Jul 20 19:27:38.256727 ip-10-0-132-219 systemd[1]: Starting Kubernetes Kubelet... Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.651798 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656832 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656843 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656847 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656850 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656856 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656860 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656863 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:38.657373 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656866 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656869 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656872 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656875 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656877 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656880 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656883 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656886 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656889 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656891 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656894 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656896 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656899 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656901 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656904 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656906 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656909 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656911 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656914 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656916 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:38.659291 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656918 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656921 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656923 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656926 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656929 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656932 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656934 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656936 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656939 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656942 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656944 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656947 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656950 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656952 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656954 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656957 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656960 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656964 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656968 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:38.672347 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656971 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656975 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656979 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656982 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656984 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656987 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656990 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656993 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656996 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.656998 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657001 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657004 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657006 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657009 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657012 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657015 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657018 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657021 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657024 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:38.825369 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657026 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:38.722749 ip-10-0-132-219 systemd[1]: Started Kubernetes Kubelet. Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657028 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657031 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657033 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657036 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657038 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657041 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657044 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657047 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657050 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657053 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657056 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657059 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657061 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657064 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657066 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657069 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657071 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657074 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:38.826315 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657077 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657079 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657082 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657084 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657087 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657090 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657092 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657094 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657097 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657100 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657102 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657105 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657107 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657109 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657112 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657798 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657804 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657807 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657812 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657817 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:38.827149 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657821 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657824 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657827 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657830 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657832 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657835 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657838 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657840 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657843 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657846 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657849 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657852 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657855 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657857 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657860 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657863 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657865 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657868 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657870 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657873 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:38.827974 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657875 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657879 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657881 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657884 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657887 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657889 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657892 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657894 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657896 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657899 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657902 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657904 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657907 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657910 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657912 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657915 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657917 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657920 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657922 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657926 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:38.828816 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657929 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657931 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657934 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657936 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657939 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657941 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657944 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657947 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657950 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657953 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657956 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657959 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657961 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657964 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657966 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657969 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657971 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657974 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657976 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:38.829458 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657979 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657982 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657984 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657986 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657989 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657992 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657994 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.657997 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658000 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658003 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658005 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658008 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658017 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658021 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658024 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658027 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658030 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658033 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658035 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:38.830069 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658038 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658041 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658043 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658046 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658048 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658051 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658053 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658056 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658058 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658061 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658064 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658067 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658070 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658072 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658075 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658077 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658466 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658472 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658475 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:38.830670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658479 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658484 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658487 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658490 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658493 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658496 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658499 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658501 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658504 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658506 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658510 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658514 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658517 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658520 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658523 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658526 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658528 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658532 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:38.831279 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658535 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658538 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658541 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658543 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658546 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658549 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658552 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658554 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658556 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658559 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658562 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658566 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658569 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658573 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658576 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658578 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658582 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658584 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658587 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658589 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:38.831906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658592 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658595 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658597 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658599 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658602 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658605 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658608 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658610 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658613 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658615 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658617 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658620 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658622 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658642 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658646 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658649 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658652 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658655 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658657 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658660 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:38.832540 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658663 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658665 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658669 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658674 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658677 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658681 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658683 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658686 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658689 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658693 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658696 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658698 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658701 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658704 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658714 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658717 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658719 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658722 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658725 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658727 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:38.833144 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658730 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658732 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658735 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658737 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658740 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658742 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658746 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658749 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658752 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658754 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658757 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658759 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658762 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658765 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658767 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658771 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658773 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.658778 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658849 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658859 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658868 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 19:27:38.833979 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658875 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658890 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658895 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658901 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658906 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658909 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658912 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658916 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658919 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658922 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658925 2580 flags.go:64] FLAG: --cgroup-root="" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658928 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658931 2580 flags.go:64] FLAG: --client-ca-file="" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658934 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658936 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658943 2580 flags.go:64] FLAG: --cluster-domain="" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658946 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658949 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658952 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658956 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658959 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658962 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658965 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 19:27:38.834579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658969 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658972 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658975 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658979 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658983 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658987 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658991 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658994 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.658997 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659000 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659003 2580 flags.go:64] FLAG: --enable-server="true" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659007 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659012 2580 flags.go:64] FLAG: --event-burst="100" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659015 2580 flags.go:64] FLAG: --event-qps="50" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659018 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659021 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659024 2580 flags.go:64] FLAG: --eviction-hard="" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659027 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659030 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659033 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659036 2580 flags.go:64] FLAG: --eviction-soft="" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659039 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659042 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659045 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659048 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 19:27:38.942133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659051 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659054 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659057 2580 flags.go:64] FLAG: --feature-gates="" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659060 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659063 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659066 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659069 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659073 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659076 2580 flags.go:64] FLAG: --help="false" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659079 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-132-219.ec2.internal" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659083 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659086 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659091 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659094 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659097 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659100 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659103 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659106 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659109 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659112 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659116 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659118 2580 flags.go:64] FLAG: --kube-reserved="" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659121 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659124 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 19:27:38.943008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659127 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659130 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659133 2580 flags.go:64] FLAG: --lock-file="" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659136 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659139 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659141 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659147 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659149 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659152 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659155 2580 flags.go:64] FLAG: --logging-format="text" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659158 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659161 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659164 2580 flags.go:64] FLAG: --manifest-url="" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659167 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659172 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659174 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659178 2580 flags.go:64] FLAG: --max-pods="110" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659181 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659186 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659189 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659193 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659196 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659199 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659202 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659209 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 19:27:38.943710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659212 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659215 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659218 2580 flags.go:64] FLAG: --pod-cidr="" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659222 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659227 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659230 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659233 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659236 2580 flags.go:64] FLAG: --port="10250" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659239 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659242 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a77a3e1996d25498" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659245 2580 flags.go:64] FLAG: --qos-reserved="" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659248 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659251 2580 flags.go:64] FLAG: --register-node="true" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659254 2580 flags.go:64] FLAG: --register-with-taints="" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659257 2580 flags.go:64] FLAG: --registry-burst="10" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659260 2580 flags.go:64] FLAG: --registry-qps="5" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659262 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659265 2580 flags.go:64] FLAG: --reserved-memory="" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659281 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659284 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659287 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659290 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659293 2580 flags.go:64] FLAG: --runonce="false" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659296 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659300 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 19:27:38.944459 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659303 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659306 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659310 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659314 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659317 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659320 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659323 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659326 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659329 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659332 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659335 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659339 2580 flags.go:64] FLAG: --system-cgroups="" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659342 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659347 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659350 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659353 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659357 2580 flags.go:64] FLAG: --tls-min-version="" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659360 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659363 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659366 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659368 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659371 2580 flags.go:64] FLAG: --v="2" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659375 2580 flags.go:64] FLAG: --version="false" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659379 2580 flags.go:64] FLAG: --vmodule="" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659382 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 19:27:38.945116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659385 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659513 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659518 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659521 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659524 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659527 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659533 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659536 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659539 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659542 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659546 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659549 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659552 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659555 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659558 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659560 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659563 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659565 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659569 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659578 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:38.945827 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659581 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659583 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659586 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659589 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659591 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659594 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659596 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659599 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659601 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659604 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659606 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659609 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659611 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659614 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659616 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659619 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659621 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659623 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659628 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659630 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:38.946383 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659633 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659635 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659639 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659641 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659646 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659649 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659652 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659655 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659658 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659661 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659664 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659667 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659675 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659678 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659681 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659685 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659688 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659691 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659693 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:38.946981 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659696 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659699 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659702 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659705 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659708 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659710 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659713 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659715 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659718 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659720 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659723 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659726 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659729 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659731 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659734 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659737 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659740 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659743 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659745 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:38.947495 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659747 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659750 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659752 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659755 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659757 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659760 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659762 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659770 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659773 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659776 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659778 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659781 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659783 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659786 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659788 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659791 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659793 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659796 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659799 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659803 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:38.948028 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659806 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.659808 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.659814 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.666201 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.666217 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.667009 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.667021 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667079 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667085 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667089 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667092 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667095 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667098 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667101 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667104 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:38.948551 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667108 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667111 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667114 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667117 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667120 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667123 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667126 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667129 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667132 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667134 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667138 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667140 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667143 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667147 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667151 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667154 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667157 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667160 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667163 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:38.948987 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667166 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667169 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667172 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667175 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667179 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667182 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667185 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667188 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667191 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667193 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667196 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667198 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667201 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667203 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667206 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667209 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667211 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667214 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667219 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667222 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:38.949549 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667225 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667227 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667230 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667232 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667235 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667238 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667240 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667243 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667245 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667248 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667251 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667254 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667257 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667259 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667262 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667264 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667267 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667269 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667272 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667275 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:38.951814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667277 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667280 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667282 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667285 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667287 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667290 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667293 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667295 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667298 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667300 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667304 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667307 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667310 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667312 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667315 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667319 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667322 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667325 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667328 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667331 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:38.952391 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667333 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667336 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667341 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667343 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667346 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667349 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667351 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667354 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667356 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667359 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667362 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667364 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.667369 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667499 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:38.976277 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667505 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667509 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667512 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667515 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667518 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667521 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667523 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667526 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667533 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667535 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667539 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667541 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667544 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667546 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667549 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667551 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667554 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667556 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667559 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667561 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:38.976791 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667563 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667567 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667570 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667572 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667575 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667577 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667580 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667582 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667585 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667588 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667590 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667593 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667595 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667598 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667601 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667603 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667606 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667608 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667610 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:38.977435 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667613 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667617 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667619 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667622 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667625 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667627 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667630 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667632 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667635 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667637 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667640 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667643 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667646 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667648 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667652 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667656 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667659 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667663 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667666 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:38.977958 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667668 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667671 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667674 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667676 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667679 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667682 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667684 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667687 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667689 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667691 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667694 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667696 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667699 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667701 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667705 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667708 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667710 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667713 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667716 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667718 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:38.978526 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667721 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667723 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667725 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667728 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667730 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667733 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667736 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667740 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667743 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667745 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667748 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667750 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667753 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667755 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667758 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667760 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667763 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667766 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667768 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:38.979514 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:38.667771 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.667776 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.667880 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.670348 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.671591 2580 server.go:1019] "Starting client certificate rotation" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.671707 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.671748 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.673917 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.677949 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.689798 2580 log.go:25] "Validated CRI v1 runtime API" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.691915 2580 log.go:25] "Validated CRI v1 image API" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.693535 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 19:27:38.980074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.696344 2580 fs.go:135] Filesystem UUIDs: map[49155d35-ddb9-4324-91f1-75cc06e70e89:/dev/nvme0n1p4 74636811-b6da-41f1-bf30-31f6b3b250ba:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 20 19:27:39.095595 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.696362 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 19:27:39.095595 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.701101 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.701586 2580 manager.go:216] Machine: {Timestamp:2026-07-20 19:27:38.700357166 +0000 UTC m=+0.325436150 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101040 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22de406dc6cb39144a60803d1c4653 SystemUUID:ec22de40-6dc6-cb39-144a-60803d1c4653 BootID:8eeeac19-5ed7-45b0-8b2a-ede44f35e235 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ab:24:ca:4f:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ab:24:ca:4f:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:a4:b5:e2:d2:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.701700 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.701778 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.704329 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.704349 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-219.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.705008 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.705018 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.705025 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.705719 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.706815 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709271 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709388 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709415 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709438 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709456 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709466 2580 kubelet.go:398] "Adding apiserver pod source" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709483 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 19:27:39.096346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709847 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.709863 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.712067 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.715524 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.715577 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717612 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717638 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717648 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717657 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717670 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717679 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717691 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717700 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717709 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717718 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717731 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.717740 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.718485 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.718498 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.721860 2580 server.go:1268] "Started kubelet" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.721937 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.721982 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.722059 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.722543 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-219.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.722681 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.722871 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-219.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.723962 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.725820 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bjb2" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.727661 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.729972 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-219.ec2.internal.18c4161b44ec6edf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-219.ec2.internal,UID:ip-10-0-132-219.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-219.ec2.internal,},FirstTimestamp:2026-07-20 19:27:38.721832671 +0000 UTC m=+0.346911655,LastTimestamp:2026-07-20 19:27:38.721832671 +0000 UTC m=+0.346911655,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-219.ec2.internal,}" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.734653 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bjb2" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.832741 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.838471 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.838481 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.838615 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.839127 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.839144 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.839263 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.839276 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.839276 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 19:27:39.097154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.839757 2580 factory.go:55] Registering systemd factory Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.839774 2580 factory.go:223] Registration of the systemd container factory successfully Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.840141 2580 factory.go:153] Registering CRI-O factory Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.840156 2580 factory.go:223] Registration of the crio container factory successfully Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.840234 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.840261 2580 factory.go:103] Registering Raw factory Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.840269 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.840278 2580 manager.go:1196] Started watching for new ooms in manager Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.840300 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-219.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.860913 2580 manager.go:349] Starting recovery of all containers Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.874200 2580 manager.go:354] Recovery completed Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.878857 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.880656 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.880676 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.880685 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.881294 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.881301 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.881315 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.881379 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.882088 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.882110 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.882119 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.882541 2580 policy_none.go:49] "None policy: Start" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.882553 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.882563 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.883578 2580 policy_none.go:47] "Start" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.933583 2580 manager.go:337] "Starting Device Plugin manager" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.933773 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.933784 2580 server.go:87] "Starting device plugin registration server" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.934009 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.934022 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.934129 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.934224 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:38.934234 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.934690 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:38.934751 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.035044 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.036061 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.036087 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.036102 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.036135 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.045617 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.045635 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-219.ec2.internal\": node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.053791 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.055017 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.055050 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 19:27:39.098337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.055069 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 19:27:39.100489 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.055099 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 19:27:39.100489 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.060303 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:39.100489 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.064714 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.219314 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.155656 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal"] Jul 20 19:27:39.219314 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.155753 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:39.219314 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.157232 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:39.219314 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.157265 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:39.219314 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.157275 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:39.219314 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.165455 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.330471 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.242395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f1c6459085a5c415e46ff01a63ff4c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal\" (UID: \"2f1c6459085a5c415e46ff01a63ff4c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.330471 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.242426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f1c6459085a5c415e46ff01a63ff4c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal\" (UID: \"2f1c6459085a5c415e46ff01a63ff4c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.330471 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.266554 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.349265 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.343331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f1c6459085a5c415e46ff01a63ff4c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal\" (UID: \"2f1c6459085a5c415e46ff01a63ff4c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.349265 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.343360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f1c6459085a5c415e46ff01a63ff4c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal\" (UID: \"2f1c6459085a5c415e46ff01a63ff4c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.349265 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.343392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f1c6459085a5c415e46ff01a63ff4c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal\" (UID: \"2f1c6459085a5c415e46ff01a63ff4c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.349265 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.343441 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f1c6459085a5c415e46ff01a63ff4c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal\" (UID: \"2f1c6459085a5c415e46ff01a63ff4c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:39.455822 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.367544 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.577575 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.468139 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.577575 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.568777 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.701048 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.669600 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.701048 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.670695 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 19:27:39.701048 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.670861 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 19:27:39.823246 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.735016 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:39.823246 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.770484 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:39.839244 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.837614 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 19:22:39 +0000 UTC" deadline="2027-12-20 14:40:09.709415517 +0000 UTC" Jul 20 19:27:39.839244 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:39.837639 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12427h12m29.87177982s" Jul 20 19:27:39.946789 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.871239 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:40.075068 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:39.971953 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:40.075068 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.027312 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:40.075068 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.072540 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-219.ec2.internal\" not found" Jul 20 19:27:40.210768 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.170937 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:40.343865 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.239604 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.343865 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.255325 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 19:27:40.705282 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.705255 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.707370 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.707353 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.710243 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.710215 2580 apiserver.go:52] "Watching apiserver" Jul 20 19:27:40.715324 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.715307 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 19:27:40.716622 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.716526 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-8f4nr","openshift-ovn-kubernetes/ovnkube-node-k2zn5","kube-system/konnectivity-agent-fx9q6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r","openshift-image-registry/node-ca-zmf8z","openshift-multus/multus-zsfcf","openshift-multus/network-metrics-daemon-b76ts","openshift-network-diagnostics/network-check-target-vs4fj","openshift-network-operator/iptables-alerter-49gfp","kube-system/global-pull-secret-syncer-dgq65","openshift-cluster-node-tuning-operator/tuned-6qjxm","openshift-dns/node-resolver-48ksk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal"] Jul 20 19:27:40.718805 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.718789 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.720765 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.720738 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 19:27:40.720765 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.720758 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 19:27:40.720964 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.720765 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 19:27:40.720964 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.720873 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-brbnm\"" Jul 20 19:27:40.721080 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.720986 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 19:27:40.721080 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.721044 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 19:27:40.721411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.721396 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.723463 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723427 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 19:27:40.723566 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723457 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 19:27:40.723566 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723483 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 19:27:40.723667 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723483 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 19:27:40.723667 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:40.723667 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723484 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jqw89\"" Jul 20 19:27:40.723817 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723671 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 19:27:40.723817 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.723689 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 19:27:40.725329 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.725315 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 19:27:40.725599 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.725584 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 19:27:40.725678 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.725586 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-94spn\"" Jul 20 19:27:40.725921 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.725908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.727711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.727660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 19:27:40.727711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.727672 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j4c8d\"" Jul 20 19:27:40.727711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.727704 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 19:27:40.727906 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.727767 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 19:27:40.728186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.728173 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.730486 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.730465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.730597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.730545 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 19:27:40.730896 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.730875 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 19:27:40.730981 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.730879 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4h977\"" Jul 20 19:27:40.731043 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.730882 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 19:27:40.731346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.731333 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.732140 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.732124 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 19:27:40.732416 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.732392 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c7nqd\"" Jul 20 19:27:40.734785 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.733992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:40.734785 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.734085 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:40.736282 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.736173 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:40.736282 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.736251 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:40.738108 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.738092 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.739103 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.739085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.740868 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.740853 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 19:27:40.741010 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.740980 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:27:40.741052 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.741014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qt5x6\"" Jul 20 19:27:40.741105 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.741084 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 19:27:40.741328 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.741314 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.741399 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.741362 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:40.743708 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.743693 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.745008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.744993 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 19:27:40.745120 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.745105 2580 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.745671 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.745658 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kpbm7\"" Jul 20 19:27:40.745724 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.745669 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:27:40.745931 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.745910 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 19:27:40.746083 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.746067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.747707 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.747691 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.748373 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.748358 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 19:27:40.748449 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.748383 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal"] Jul 20 19:27:40.748740 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.748712 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 19:27:40.748930 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.748917 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 19:27:40.749180 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.749164 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sqmkm\"" Jul 20 19:27:40.751828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.751808 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-netns\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.752004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.751834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-os-release\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.752004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.751852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-conf-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.752004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.751869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-ovn\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.752004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.751895 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-iptables-alerter-script\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.752004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.751946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.752004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.751991 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-node-log\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-ovnkube-script-lib\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-k8s-cni-cncf-io\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752076 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-dbus\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-etc-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752147 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-run-ovn-kubernetes\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-cni-netd\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c22b57f42ff2be1a90bb5b7bd856167f-config\") pod \"kube-apiserver-proxy-ip-10-0-132-219.ec2.internal\" (UID: \"c22b57f42ff2be1a90bb5b7bd856167f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a960bab6-ebec-418e-b780-1294ccf3b592-konnectivity-ca\") pod \"konnectivity-agent-fx9q6\" (UID: \"a960bab6-ebec-418e-b780-1294ccf3b592\") " pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-etc-selinux\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b410358b-9644-4059-8c56-a03ab134ebca-ovn-node-metrics-cert\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-system-cni-dir\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.752388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-daemon-config\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmbsj\" (UniqueName: \"kubernetes.io/projected/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-kube-api-access-xmbsj\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-cnibin\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752495 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752549 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a960bab6-ebec-418e-b780-1294ccf3b592-agent-certs\") pod \"konnectivity-agent-fx9q6\" (UID: \"a960bab6-ebec-418e-b780-1294ccf3b592\") " pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-device-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fa9cad5-2584-4972-9ab0-47dd94309bdc-host\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752638 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-cni-bin\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752668 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-etc-kubernetes\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-kubelet-config\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnwx7\" (UniqueName: \"kubernetes.io/projected/98b76723-379b-49eb-ae59-2d6122a50121-kube-api-access-jnwx7\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v7zl\" (UniqueName: \"kubernetes.io/projected/5fa9cad5-2584-4972-9ab0-47dd94309bdc-kube-api-access-4v7zl\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752941 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-cnibin\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.752971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-systemd-units\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-env-overrides\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.753036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-sys-fs\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fa9cad5-2584-4972-9ab0-47dd94309bdc-serviceca\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-socket-dir-parent\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-run-netns\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753232 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-systemd\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753308 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-socket-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-registration-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753376 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-system-cni-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df37270e-b6d6-4df9-9752-78889f3c10f2-cni-binary-copy\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-cni-multus\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-kubelet\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7cdd\" (UniqueName: \"kubernetes.io/projected/df37270e-b6d6-4df9-9752-78889f3c10f2-kube-api-access-b7cdd\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753556 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldwg8\" (UniqueName: \"kubernetes.io/projected/65a53832-7082-4db5-80fb-2df330fb6f2f-kube-api-access-ldwg8\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-776r2\" (UniqueName: \"kubernetes.io/projected/b410358b-9644-4059-8c56-a03ab134ebca-kube-api-access-776r2\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.754680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753657 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8scfm\" (UniqueName: \"kubernetes.io/projected/a7cfe41a-907b-49d4-89c1-be02103ea048-kube-api-access-8scfm\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-kubelet\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-multus-certs\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-host-slash\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753811 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-os-release\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753871 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-slash\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-cni-bin\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753949 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-hostroot\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-var-lib-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.753992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-ovnkube-config\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.754014 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.754037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-cni-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.754059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.755186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.754080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-log-socket\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.839576 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.839554 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 19:27:40.840046 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.840026 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 19:27:40.854388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-cni-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.854519 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.854519 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-log-socket\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.854519 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-kubernetes\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.854519 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854500 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-netns\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-modprobe-d\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysctl-d\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.854553 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854563 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-log-socket\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854596 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-netns\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.854632 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret podName:c1e1ab83-94d8-4a81-9d3f-c14b59e8e542 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:41.354615199 +0000 UTC m=+2.979694174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret") pod "global-pull-secret-syncer-dgq65" (UID: "c1e1ab83-94d8-4a81-9d3f-c14b59e8e542") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-os-release\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.854687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-conf-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-ovn\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854700 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-cni-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-tuned\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854719 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-os-release\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-iptables-alerter-script\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854748 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-conf-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-ovn\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854806 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-node-log\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-ovnkube-script-lib\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-node-log\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-host\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.855013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.854974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-k8s-cni-cncf-io\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855041 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-k8s-cni-cncf-io\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-dbus\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-etc-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-run-ovn-kubernetes\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855136 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-dbus\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855171 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-etc-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-cni-netd\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855193 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-run-ovn-kubernetes\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855248 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-cni-netd\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c22b57f42ff2be1a90bb5b7bd856167f-config\") pod \"kube-apiserver-proxy-ip-10-0-132-219.ec2.internal\" (UID: \"c22b57f42ff2be1a90bb5b7bd856167f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a960bab6-ebec-418e-b780-1294ccf3b592-konnectivity-ca\") pod \"konnectivity-agent-fx9q6\" (UID: \"a960bab6-ebec-418e-b780-1294ccf3b592\") " pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855309 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c22b57f42ff2be1a90bb5b7bd856167f-config\") pod \"kube-apiserver-proxy-ip-10-0-132-219.ec2.internal\" (UID: \"c22b57f42ff2be1a90bb5b7bd856167f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-etc-selinux\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b410358b-9644-4059-8c56-a03ab134ebca-ovn-node-metrics-cert\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855368 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855460 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-system-cni-dir\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.855597 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.855466 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855505 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-etc-selinux\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-ovnkube-script-lib\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.855527 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:41.355508556 +0000 UTC m=+2.980587545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855553 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-run\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-daemon-config\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmbsj\" (UniqueName: \"kubernetes.io/projected/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-kube-api-access-xmbsj\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-cnibin\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a960bab6-ebec-418e-b780-1294ccf3b592-agent-certs\") pod \"konnectivity-agent-fx9q6\" (UID: \"a960bab6-ebec-418e-b780-1294ccf3b592\") " pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-cnibin\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-device-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fa9cad5-2584-4972-9ab0-47dd94309bdc-host\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855768 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 19:27:40.856559 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40ce65db-b05a-4930-927c-b81a075191df-hosts-file\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysctl-conf\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-sys\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-cni-bin\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855874 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855877 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-etc-kubernetes\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855919 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-etc-kubernetes\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-kubelet-config\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855953 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a960bab6-ebec-418e-b780-1294ccf3b592-konnectivity-ca\") pod \"konnectivity-agent-fx9q6\" (UID: \"a960bab6-ebec-418e-b780-1294ccf3b592\") " pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnwx7\" (UniqueName: \"kubernetes.io/projected/98b76723-379b-49eb-ae59-2d6122a50121-kube-api-access-jnwx7\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v7zl\" (UniqueName: \"kubernetes.io/projected/5fa9cad5-2584-4972-9ab0-47dd94309bdc-kube-api-access-4v7zl\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856010 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-cni-bin\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-kubelet-config\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856022 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysconfig\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-cnibin\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-systemd-units\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856135 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-iptables-alerter-script\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.857441 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-env-overrides\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856155 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-daemon-config\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-sys-fs\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fa9cad5-2584-4972-9ab0-47dd94309bdc-serviceca\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856222 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-sys-fs\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-lib-modules\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856223 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-cnibin\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856276 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-socket-dir-parent\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.855593 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-system-cni-dir\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fa9cad5-2584-4972-9ab0-47dd94309bdc-host\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-run-netns\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-systemd\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856424 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-systemd-units\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856505 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856532 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-socket-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856557 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-registration-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856584 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-systemd\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-system-cni-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856614 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-systemd\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856634 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-env-overrides\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856670 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df37270e-b6d6-4df9-9752-78889f3c10f2-cni-binary-copy\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-cni-multus\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856740 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-socket-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-kubelet\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7cdd\" (UniqueName: \"kubernetes.io/projected/df37270e-b6d6-4df9-9752-78889f3c10f2-kube-api-access-b7cdd\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-run-netns\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwg8\" (UniqueName: \"kubernetes.io/projected/65a53832-7082-4db5-80fb-2df330fb6f2f-kube-api-access-ldwg8\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856826 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jq42\" (UniqueName: \"kubernetes.io/projected/48b1bd84-9635-4cc1-8397-35f3c896d071-kube-api-access-7jq42\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-system-cni-dir\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztb6h\" (UniqueName: \"kubernetes.io/projected/40ce65db-b05a-4930-927c-b81a075191df-kube-api-access-ztb6h\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856880 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fa9cad5-2584-4972-9ab0-47dd94309bdc-serviceca\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-776r2\" (UniqueName: \"kubernetes.io/projected/b410358b-9644-4059-8c56-a03ab134ebca-kube-api-access-776r2\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.858733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8scfm\" (UniqueName: \"kubernetes.io/projected/a7cfe41a-907b-49d4-89c1-be02103ea048-kube-api-access-8scfm\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856985 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-multus-socket-dir-parent\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.856944 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-device-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-kubelet\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857064 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-kubelet\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857082 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65a53832-7082-4db5-80fb-2df330fb6f2f-registration-dir\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-var-lib-kubelet\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-multus-certs\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-host-slash\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857187 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-kubelet\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-os-release\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-host-slash\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-slash\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857282 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-slash\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857282 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-run-multus-certs\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859217 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857318 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df37270e-b6d6-4df9-9752-78889f3c10f2-cni-binary-copy\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857333 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98b76723-379b-49eb-ae59-2d6122a50121-os-release\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-host-var-lib-cni-multus\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-cni-bin\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857376 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-run-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857407 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-host-cni-bin\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-hostroot\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-var-lib-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-ovnkube-config\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857556 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b410358b-9644-4059-8c56-a03ab134ebca-var-lib-openvswitch\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ce65db-b05a-4930-927c-b81a075191df-tmp-dir\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857564 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df37270e-b6d6-4df9-9752-78889f3c10f2-hostroot\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48b1bd84-9635-4cc1-8397-35f3c896d071-tmp\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.857971 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/98b76723-379b-49eb-ae59-2d6122a50121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.858033 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b410358b-9644-4059-8c56-a03ab134ebca-ovnkube-config\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.858130 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 19:27:40.859974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.859544 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b410358b-9644-4059-8c56-a03ab134ebca-ovn-node-metrics-cert\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.860463 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.859715 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a960bab6-ebec-418e-b780-1294ccf3b592-agent-certs\") pod \"konnectivity-agent-fx9q6\" (UID: \"a960bab6-ebec-418e-b780-1294ccf3b592\") " pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:40.870929 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.870899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnwx7\" (UniqueName: \"kubernetes.io/projected/98b76723-379b-49eb-ae59-2d6122a50121-kube-api-access-jnwx7\") pod \"multus-additional-cni-plugins-8f4nr\" (UID: \"98b76723-379b-49eb-ae59-2d6122a50121\") " pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:40.871743 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.871717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v7zl\" (UniqueName: \"kubernetes.io/projected/5fa9cad5-2584-4972-9ab0-47dd94309bdc-kube-api-access-4v7zl\") pod \"node-ca-zmf8z\" (UID: \"5fa9cad5-2584-4972-9ab0-47dd94309bdc\") " pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:40.872174 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.872150 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmbsj\" (UniqueName: \"kubernetes.io/projected/1536be0d-fe2e-4ee0-b8d5-71437163ac9d-kube-api-access-xmbsj\") pod \"iptables-alerter-49gfp\" (UID: \"1536be0d-fe2e-4ee0-b8d5-71437163ac9d\") " pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:40.872611 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.872590 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-776r2\" (UniqueName: \"kubernetes.io/projected/b410358b-9644-4059-8c56-a03ab134ebca-kube-api-access-776r2\") pod \"ovnkube-node-k2zn5\" (UID: \"b410358b-9644-4059-8c56-a03ab134ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:40.872975 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.872955 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:40.872975 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.872970 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldwg8\" (UniqueName: \"kubernetes.io/projected/65a53832-7082-4db5-80fb-2df330fb6f2f-kube-api-access-ldwg8\") pod \"aws-ebs-csi-driver-node-whv8r\" (UID: \"65a53832-7082-4db5-80fb-2df330fb6f2f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:40.873117 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.872982 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:40.873117 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.872997 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pdq4x for pod openshift-network-diagnostics/network-check-target-vs4fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:40.873117 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:40.873068 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x podName:03d18b33-38dc-49c5-9c9b-ef12ac983df3 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:41.373050265 +0000 UTC m=+2.998129254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pdq4x" (UniqueName: "kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x") pod "network-check-target-vs4fj" (UID: "03d18b33-38dc-49c5-9c9b-ef12ac983df3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:40.873272 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.873142 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8scfm\" (UniqueName: \"kubernetes.io/projected/a7cfe41a-907b-49d4-89c1-be02103ea048-kube-api-access-8scfm\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:40.873771 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.873750 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7cdd\" (UniqueName: \"kubernetes.io/projected/df37270e-b6d6-4df9-9752-78889f3c10f2-kube-api-access-b7cdd\") pod \"multus-zsfcf\" (UID: \"df37270e-b6d6-4df9-9752-78889f3c10f2\") " pod="openshift-multus/multus-zsfcf" Jul 20 19:27:40.904654 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.904633 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hpjtf" Jul 20 19:27:40.910823 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.910809 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hpjtf" Jul 20 19:27:40.958976 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.958937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-lib-modules\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.958976 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.958978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-systemd\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959151 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jq42\" (UniqueName: \"kubernetes.io/projected/48b1bd84-9635-4cc1-8397-35f3c896d071-kube-api-access-7jq42\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959195 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-lib-modules\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959249 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztb6h\" (UniqueName: \"kubernetes.io/projected/40ce65db-b05a-4930-927c-b81a075191df-kube-api-access-ztb6h\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.959249 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-systemd\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959350 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-var-lib-kubelet\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959403 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959375 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-var-lib-kubelet\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959403 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ce65db-b05a-4930-927c-b81a075191df-tmp-dir\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.959537 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48b1bd84-9635-4cc1-8397-35f3c896d071-tmp\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959537 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-kubernetes\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959537 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-modprobe-d\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959537 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959500 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysctl-d\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959537 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959518 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-tuned\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-host\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-run\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959635 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysctl-d\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-modprobe-d\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959682 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ce65db-b05a-4930-927c-b81a075191df-tmp-dir\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.959781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959725 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-run\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.959781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959725 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-host\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.960124 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40ce65db-b05a-4930-927c-b81a075191df-hosts-file\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.960124 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959891 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40ce65db-b05a-4930-927c-b81a075191df-hosts-file\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.960124 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.959921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysctl-conf\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.960124 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.960080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-sys\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.960124 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.960094 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysctl-conf\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.960366 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.960152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysconfig\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.960366 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.960162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-sys\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.960366 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.960168 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-kubernetes\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.960366 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.960289 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-sysconfig\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.961881 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.961861 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48b1bd84-9635-4cc1-8397-35f3c896d071-tmp\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.961970 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.961914 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48b1bd84-9635-4cc1-8397-35f3c896d071-etc-tuned\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:40.969870 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.969851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztb6h\" (UniqueName: \"kubernetes.io/projected/40ce65db-b05a-4930-927c-b81a075191df-kube-api-access-ztb6h\") pod \"node-resolver-48ksk\" (UID: \"40ce65db-b05a-4930-927c-b81a075191df\") " pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:40.969959 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:40.969922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jq42\" (UniqueName: \"kubernetes.io/projected/48b1bd84-9635-4cc1-8397-35f3c896d071-kube-api-access-7jq42\") pod \"tuned-6qjxm\" (UID: \"48b1bd84-9635-4cc1-8397-35f3c896d071\") " pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:41.052025 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.051935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" Jul 20 19:27:41.057336 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.057293 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:41.066959 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.066939 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" Jul 20 19:27:41.088882 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.088861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:27:41.103075 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.103044 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:27:41.122303 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.122275 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" Jul 20 19:27:41.135032 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.135012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmf8z" Jul 20 19:27:41.141021 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.140994 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zsfcf" Jul 20 19:27:41.172007 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.171983 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49gfp" Jul 20 19:27:41.195948 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.195930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" Jul 20 19:27:41.200761 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.200746 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-48ksk" Jul 20 19:27:41.362364 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.362280 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:41.362364 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.362340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:41.362622 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.362435 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:41.362622 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.362462 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:41.362622 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.362503 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:42.362488349 +0000 UTC m=+3.987567322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:41.362622 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.362517 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret podName:c1e1ab83-94d8-4a81-9d3f-c14b59e8e542 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:42.362510678 +0000 UTC m=+3.987589651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret") pod "global-pull-secret-syncer-dgq65" (UID: "c1e1ab83-94d8-4a81-9d3f-c14b59e8e542") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:41.463344 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.463316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:41.463515 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.463442 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:41.463515 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.463476 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:41.463515 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.463488 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pdq4x for pod openshift-network-diagnostics/network-check-target-vs4fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:41.463616 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:41.463544 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x podName:03d18b33-38dc-49c5-9c9b-ef12ac983df3 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:42.463528614 +0000 UTC m=+4.088607589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pdq4x" (UniqueName: "kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x") pod "network-check-target-vs4fj" (UID: "03d18b33-38dc-49c5-9c9b-ef12ac983df3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:41.786351 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.786308 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22b57f42ff2be1a90bb5b7bd856167f.slice/crio-916476424ff5712ea50a38fde513680837bcfd6f5ae0a8a774f1a41835ab8ac4 WatchSource:0}: Error finding container 916476424ff5712ea50a38fde513680837bcfd6f5ae0a8a774f1a41835ab8ac4: Status 404 returned error can't find the container with id 916476424ff5712ea50a38fde513680837bcfd6f5ae0a8a774f1a41835ab8ac4 Jul 20 19:27:41.793327 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.793306 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 19:27:41.805195 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.805171 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1c6459085a5c415e46ff01a63ff4c0.slice/crio-2012fb7f5ed85a40101c41fd367fa85b59dae94ca189f7ffbe24d143c930efbb WatchSource:0}: Error finding container 2012fb7f5ed85a40101c41fd367fa85b59dae94ca189f7ffbe24d143c930efbb: Status 404 returned error can't find the container with id 2012fb7f5ed85a40101c41fd367fa85b59dae94ca189f7ffbe24d143c930efbb Jul 20 19:27:41.815900 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.815872 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b1bd84_9635_4cc1_8397_35f3c896d071.slice/crio-d25e368974432c159e1d33cf8879184945798a0b31853bd157e499d7e2e4174d WatchSource:0}: Error finding container d25e368974432c159e1d33cf8879184945798a0b31853bd157e499d7e2e4174d: Status 404 returned error can't find the container with id d25e368974432c159e1d33cf8879184945798a0b31853bd157e499d7e2e4174d Jul 20 19:27:41.835917 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.835894 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb410358b_9644_4059_8c56_a03ab134ebca.slice/crio-f3d5b27dadcab19e2c023f7ca08d57b45651ea587279ea517224b7fb4d4f9c18 WatchSource:0}: Error finding container f3d5b27dadcab19e2c023f7ca08d57b45651ea587279ea517224b7fb4d4f9c18: Status 404 returned error can't find the container with id f3d5b27dadcab19e2c023f7ca08d57b45651ea587279ea517224b7fb4d4f9c18 Jul 20 19:27:41.870139 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.870115 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1536be0d_fe2e_4ee0_b8d5_71437163ac9d.slice/crio-8c57c57bd2bd02be9c6e2148f2c09e189203747091d6be57cf7fb4c2396ba453 WatchSource:0}: Error finding container 8c57c57bd2bd02be9c6e2148f2c09e189203747091d6be57cf7fb4c2396ba453: Status 404 returned error can't find the container with id 8c57c57bd2bd02be9c6e2148f2c09e189203747091d6be57cf7fb4c2396ba453 Jul 20 19:27:41.891615 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.891590 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a53832_7082_4db5_80fb_2df330fb6f2f.slice/crio-4ee7ecc361c926a2faa33a0c91ac908d9b01dcd3bb18000a551f3688f97dbcfc WatchSource:0}: Error finding container 4ee7ecc361c926a2faa33a0c91ac908d9b01dcd3bb18000a551f3688f97dbcfc: Status 404 returned error can't find the container with id 4ee7ecc361c926a2faa33a0c91ac908d9b01dcd3bb18000a551f3688f97dbcfc Jul 20 19:27:41.898645 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.898609 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ce65db_b05a_4930_927c_b81a075191df.slice/crio-1d0bf8de0d5000f78dde33725d4448a8d543667a112f1f1506fb9a3bf7bd0308 WatchSource:0}: Error finding container 1d0bf8de0d5000f78dde33725d4448a8d543667a112f1f1506fb9a3bf7bd0308: Status 404 returned error can't find the container with id 1d0bf8de0d5000f78dde33725d4448a8d543667a112f1f1506fb9a3bf7bd0308 Jul 20 19:27:41.915834 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.915802 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 19:22:41 +0000 UTC" deadline="2028-02-01 00:56:13.387381968 +0000 UTC" Jul 20 19:27:41.915834 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:41.915831 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13445h28m31.471554549s" Jul 20 19:27:41.944626 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.944594 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa9cad5_2584_4972_9ab0_47dd94309bdc.slice/crio-de5155fb1155f73742e0f66dddef065063071ea3f6fd5b7c8ccdefdf9b2798ca WatchSource:0}: Error finding container de5155fb1155f73742e0f66dddef065063071ea3f6fd5b7c8ccdefdf9b2798ca: Status 404 returned error can't find the container with id de5155fb1155f73742e0f66dddef065063071ea3f6fd5b7c8ccdefdf9b2798ca Jul 20 19:27:41.957335 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.957311 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf37270e_b6d6_4df9_9752_78889f3c10f2.slice/crio-f5c9aaab56e7bebd536bd458088f19950eea437ec9d412175d66122427726997 WatchSource:0}: Error finding container f5c9aaab56e7bebd536bd458088f19950eea437ec9d412175d66122427726997: Status 404 returned error can't find the container with id f5c9aaab56e7bebd536bd458088f19950eea437ec9d412175d66122427726997 Jul 20 19:27:41.977543 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:41.977522 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda960bab6_ebec_418e_b780_1294ccf3b592.slice/crio-2c142834dea3b4114aa3bf74eeeea34e9646cabe45798b0928b94877e0e2c670 WatchSource:0}: Error finding container 2c142834dea3b4114aa3bf74eeeea34e9646cabe45798b0928b94877e0e2c670: Status 404 returned error can't find the container with id 2c142834dea3b4114aa3bf74eeeea34e9646cabe45798b0928b94877e0e2c670 Jul 20 19:27:42.056151 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.056077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:42.056151 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.056115 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:42.056316 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.056190 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:42.056357 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.056336 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:42.061850 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.061809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-48ksk" event={"ID":"40ce65db-b05a-4930-927c-b81a075191df","Type":"ContainerStarted","Data":"1d0bf8de0d5000f78dde33725d4448a8d543667a112f1f1506fb9a3bf7bd0308"} Jul 20 19:27:42.062959 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.062935 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49gfp" event={"ID":"1536be0d-fe2e-4ee0-b8d5-71437163ac9d","Type":"ContainerStarted","Data":"8c57c57bd2bd02be9c6e2148f2c09e189203747091d6be57cf7fb4c2396ba453"} Jul 20 19:27:42.064026 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.064005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"f3d5b27dadcab19e2c023f7ca08d57b45651ea587279ea517224b7fb4d4f9c18"} Jul 20 19:27:42.065050 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.065029 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" event={"ID":"2f1c6459085a5c415e46ff01a63ff4c0","Type":"ContainerStarted","Data":"2012fb7f5ed85a40101c41fd367fa85b59dae94ca189f7ffbe24d143c930efbb"} Jul 20 19:27:42.066049 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.066027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" event={"ID":"c22b57f42ff2be1a90bb5b7bd856167f","Type":"ContainerStarted","Data":"916476424ff5712ea50a38fde513680837bcfd6f5ae0a8a774f1a41835ab8ac4"} Jul 20 19:27:42.068169 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.067288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zsfcf" event={"ID":"df37270e-b6d6-4df9-9752-78889f3c10f2","Type":"ContainerStarted","Data":"f5c9aaab56e7bebd536bd458088f19950eea437ec9d412175d66122427726997"} Jul 20 19:27:42.069762 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.069740 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" event={"ID":"65a53832-7082-4db5-80fb-2df330fb6f2f","Type":"ContainerStarted","Data":"4ee7ecc361c926a2faa33a0c91ac908d9b01dcd3bb18000a551f3688f97dbcfc"} Jul 20 19:27:42.070748 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.070726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" event={"ID":"48b1bd84-9635-4cc1-8397-35f3c896d071","Type":"ContainerStarted","Data":"d25e368974432c159e1d33cf8879184945798a0b31853bd157e499d7e2e4174d"} Jul 20 19:27:42.071773 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.071748 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fx9q6" event={"ID":"a960bab6-ebec-418e-b780-1294ccf3b592","Type":"ContainerStarted","Data":"2c142834dea3b4114aa3bf74eeeea34e9646cabe45798b0928b94877e0e2c670"} Jul 20 19:27:42.073048 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.073029 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmf8z" event={"ID":"5fa9cad5-2584-4972-9ab0-47dd94309bdc","Type":"ContainerStarted","Data":"de5155fb1155f73742e0f66dddef065063071ea3f6fd5b7c8ccdefdf9b2798ca"} Jul 20 19:27:42.091264 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:27:42.091245 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b76723_379b_49eb_ae59_2d6122a50121.slice/crio-d2dc9f8185ec7f0a5c4507df52ab7a70999aca8a596f656d1181cff1bb010b8e WatchSource:0}: Error finding container d2dc9f8185ec7f0a5c4507df52ab7a70999aca8a596f656d1181cff1bb010b8e: Status 404 returned error can't find the container with id d2dc9f8185ec7f0a5c4507df52ab7a70999aca8a596f656d1181cff1bb010b8e Jul 20 19:27:42.370322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.370225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:42.370322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.370290 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:42.370580 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.370429 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:42.370639 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.370611 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:44.370562718 +0000 UTC m=+5.995641709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:42.371046 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.371024 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:42.371151 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.371083 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret podName:c1e1ab83-94d8-4a81-9d3f-c14b59e8e542 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:44.371067885 +0000 UTC m=+5.996146864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret") pod "global-pull-secret-syncer-dgq65" (UID: "c1e1ab83-94d8-4a81-9d3f-c14b59e8e542") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:42.471688 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.471639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:42.471880 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.471835 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:42.471880 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.471857 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:42.471880 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.471871 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pdq4x for pod openshift-network-diagnostics/network-check-target-vs4fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:42.472102 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:42.471931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x podName:03d18b33-38dc-49c5-9c9b-ef12ac983df3 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:44.471912728 +0000 UTC m=+6.096991708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pdq4x" (UniqueName: "kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x") pod "network-check-target-vs4fj" (UID: "03d18b33-38dc-49c5-9c9b-ef12ac983df3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:42.916742 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.916662 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 19:22:41 +0000 UTC" deadline="2028-03-26 14:10:59.282867249 +0000 UTC" Jul 20 19:27:42.916742 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:42.916712 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14754h43m16.366160484s" Jul 20 19:27:43.058659 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:43.058629 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:43.058842 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:43.058765 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:43.094854 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:43.094811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerStarted","Data":"d2dc9f8185ec7f0a5c4507df52ab7a70999aca8a596f656d1181cff1bb010b8e"} Jul 20 19:27:44.056094 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:44.056047 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:44.056094 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:44.056079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:44.056633 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.056184 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:44.056633 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.056314 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:44.387547 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:44.386772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:44.387547 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:44.386828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:44.387547 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.386953 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:44.387547 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.387012 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:48.386994455 +0000 UTC m=+10.012073440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:44.387547 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.387396 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:44.387547 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.387465 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret podName:c1e1ab83-94d8-4a81-9d3f-c14b59e8e542 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:48.387449033 +0000 UTC m=+10.012528009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret") pod "global-pull-secret-syncer-dgq65" (UID: "c1e1ab83-94d8-4a81-9d3f-c14b59e8e542") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:44.488638 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:44.488601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:44.488889 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.488869 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:44.488972 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.488901 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:44.488972 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.488913 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pdq4x for pod openshift-network-diagnostics/network-check-target-vs4fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:44.489065 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:44.488997 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x podName:03d18b33-38dc-49c5-9c9b-ef12ac983df3 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:48.488952775 +0000 UTC m=+10.114031749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pdq4x" (UniqueName: "kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x") pod "network-check-target-vs4fj" (UID: "03d18b33-38dc-49c5-9c9b-ef12ac983df3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:45.056719 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:45.056656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:45.057186 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:45.056829 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:46.056612 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:46.055904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:46.056612 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:46.056045 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:46.056612 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:46.056461 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:46.056612 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:46.056560 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:47.056266 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:47.056235 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:47.056419 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:47.056357 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:48.056249 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:48.056212 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:48.056717 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:48.056274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:48.056717 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.056403 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:48.056717 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.056556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:48.420549 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:48.420166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:48.420549 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:48.420232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:48.420549 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.420375 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:48.420549 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.420401 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:48.420549 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.420459 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:56.420438814 +0000 UTC m=+18.045517802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:48.420549 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.420486 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret podName:c1e1ab83-94d8-4a81-9d3f-c14b59e8e542 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:56.420466576 +0000 UTC m=+18.045545565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret") pod "global-pull-secret-syncer-dgq65" (UID: "c1e1ab83-94d8-4a81-9d3f-c14b59e8e542") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:48.521762 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:48.521125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:48.521762 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.521311 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:48.521762 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.521331 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:48.521762 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.521346 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pdq4x for pod openshift-network-diagnostics/network-check-target-vs4fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:48.521762 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:48.521414 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x podName:03d18b33-38dc-49c5-9c9b-ef12ac983df3 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:56.521394794 +0000 UTC m=+18.146473780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pdq4x" (UniqueName: "kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x") pod "network-check-target-vs4fj" (UID: "03d18b33-38dc-49c5-9c9b-ef12ac983df3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:49.056208 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:49.056175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:49.056395 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:49.056305 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:50.055587 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:50.055552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:50.055775 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:50.055717 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:50.055775 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:50.055557 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:50.055902 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:50.055838 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:51.056538 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:51.055630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:51.056538 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:51.055772 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:52.055471 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:52.055366 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:52.055647 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:52.055514 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:52.055720 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:52.055705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:52.055870 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:52.055844 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:53.055383 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:53.055346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:53.055817 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:53.055488 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:54.056115 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:54.056038 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:54.056530 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:54.056042 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:54.056530 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:54.056155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:54.056530 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:54.056288 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:55.056231 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:55.056195 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:55.056723 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:55.056327 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:56.055685 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:56.055650 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:56.055880 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.055791 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:56.055880 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:56.055837 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:56.055992 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.055949 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:56.480887 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:56.480797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:56.480887 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:56.480860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:56.481354 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.480964 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:56.481354 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.480982 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:56.481354 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.481046 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret podName:c1e1ab83-94d8-4a81-9d3f-c14b59e8e542 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:12.48102571 +0000 UTC m=+34.106104682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret") pod "global-pull-secret-syncer-dgq65" (UID: "c1e1ab83-94d8-4a81-9d3f-c14b59e8e542") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:56.481354 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.481068 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:12.481058513 +0000 UTC m=+34.106137487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:56.581924 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:56.581888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:56.582097 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.582019 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:56.582097 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.582038 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:56.582097 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.582050 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pdq4x for pod openshift-network-diagnostics/network-check-target-vs4fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:56.582228 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:56.582105 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x podName:03d18b33-38dc-49c5-9c9b-ef12ac983df3 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:12.582091887 +0000 UTC m=+34.207170861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pdq4x" (UniqueName: "kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x") pod "network-check-target-vs4fj" (UID: "03d18b33-38dc-49c5-9c9b-ef12ac983df3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:57.056032 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:57.055997 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:57.056222 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:57.056137 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:27:58.055376 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:58.055346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:27:58.055851 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:58.055346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:27:58.055851 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:58.055487 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:27:58.055851 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:58.055557 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:27:59.058489 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:27:59.058455 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:27:59.058945 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:27:59.058575 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:00.055608 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:00.055568 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:00.055798 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:00.055704 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:00.055798 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:00.055712 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:00.055911 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:00.055839 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:01.058629 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:01.058601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:01.058990 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:01.058702 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:02.056814 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.056589 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:02.057625 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:02.057279 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:02.057889 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.057640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:02.058414 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:02.058393 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:02.144138 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.143964 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"d1e2907eb22393079ec7d639bd53e541c2c78e0d06fadd9290ae4b644fa57a6b"} Jul 20 19:28:02.144138 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.144066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"c6f26a7d549cfed38f995e5d9f4c28b7e526da7d07780a1ef44f98ae1e3e63b0"} Jul 20 19:28:02.151082 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.150366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" event={"ID":"c22b57f42ff2be1a90bb5b7bd856167f","Type":"ContainerStarted","Data":"ee06c48c87c1a7a18288746f3cad82531036f05da68054470809b0a4e18243f6"} Jul 20 19:28:02.157158 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.157103 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zsfcf" event={"ID":"df37270e-b6d6-4df9-9752-78889f3c10f2","Type":"ContainerStarted","Data":"a37a46f5b2b227c7b7339ab44a7710709813a946632ee469c5a0315a361c35a1"} Jul 20 19:28:02.159001 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.158980 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" event={"ID":"48b1bd84-9635-4cc1-8397-35f3c896d071","Type":"ContainerStarted","Data":"0747a839be13a5ebc6d035ae5c8ad2972d13651cb62cfd253d6fbe05c2ad2290"} Jul 20 19:28:02.166315 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.166275 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-219.ec2.internal" podStartSLOduration=22.166261378 podStartE2EDuration="22.166261378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:02.165906541 +0000 UTC m=+23.790985534" watchObservedRunningTime="2026-07-20 19:28:02.166261378 +0000 UTC m=+23.791340364" Jul 20 19:28:02.196627 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.196567 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zsfcf" podStartSLOduration=3.186678982 podStartE2EDuration="23.196551565s" totalImagesPullingTime="20.009872583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:02.196213828 +0000 UTC m=+23.821292833" watchObservedRunningTime="2026-07-20 19:28:02.196551565 +0000 UTC m=+23.821630560" Jul 20 19:28:02.196898 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:02.196871 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6qjxm" podStartSLOduration=3.239538277 podStartE2EDuration="23.196863715s" totalImagesPullingTime="19.957325438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:02.180937812 +0000 UTC m=+23.806016807" watchObservedRunningTime="2026-07-20 19:28:02.196863715 +0000 UTC m=+23.821942709" Jul 20 19:28:03.056305 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.056275 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:03.056492 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:03.056378 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:03.163264 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.163225 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" event={"ID":"65a53832-7082-4db5-80fb-2df330fb6f2f","Type":"ContainerStarted","Data":"c2fcdb964ac3943358167ad55e0162d22f3dddf88b12e9349edbd3841142c7ba"} Jul 20 19:28:03.164977 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.164943 2580 generic.go:356] "Generic (PLEG): container finished" podID="98b76723-379b-49eb-ae59-2d6122a50121" containerID="cdaeed041100f531e49f368c432334c961b2dda82d49c02c0eb99701fef9c539" exitCode=0 Jul 20 19:28:03.165092 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.165025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerDied","Data":"cdaeed041100f531e49f368c432334c961b2dda82d49c02c0eb99701fef9c539"} Jul 20 19:28:03.170383 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.170082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fx9q6" event={"ID":"a960bab6-ebec-418e-b780-1294ccf3b592","Type":"ContainerStarted","Data":"32f9608ff59c0e7e346f0e7f3904969696d1b29af45f33bb5becb105737118aa"} Jul 20 19:28:03.172290 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.172256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmf8z" event={"ID":"5fa9cad5-2584-4972-9ab0-47dd94309bdc","Type":"ContainerStarted","Data":"8e7ba947e886f56467da848f53ac47652d091d670085c1955ddc6b797536133b"} Jul 20 19:28:03.173986 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.173963 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-48ksk" event={"ID":"40ce65db-b05a-4930-927c-b81a075191df","Type":"ContainerStarted","Data":"6e103a9b92bd3747458b9d0c283ae8fb80ac52300e21bad7e8a10ff9060a1a3b"} Jul 20 19:28:03.177622 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.177600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"8ee9aec5efa63b2ebd08c5d30b7211a7606dba8a954a394026382352d17a1f3f"} Jul 20 19:28:03.177730 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.177664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"1d903a25822b90cbff6a164f7bca159e39529356827a96d0440085553fb11a31"} Jul 20 19:28:03.177730 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.177708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"31c1d795f9f25fb2ac86a566f547d1febbcc6b88104cd597d8576ebd81dde863"} Jul 20 19:28:03.177847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.177748 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"524ddbe1ed7afe739de59ba5d41eb9a24eadbd6cf620e9e7a7d510cf05406621"} Jul 20 19:28:03.179097 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.179075 2580 generic.go:356] "Generic (PLEG): container finished" podID="2f1c6459085a5c415e46ff01a63ff4c0" containerID="11cb840fefb3fd861b573d1f52735499c5b0b4c51ef6cbf40382c4b9a932bd07" exitCode=0 Jul 20 19:28:03.179212 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.179180 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" event={"ID":"2f1c6459085a5c415e46ff01a63ff4c0","Type":"ContainerDied","Data":"11cb840fefb3fd861b573d1f52735499c5b0b4c51ef6cbf40382c4b9a932bd07"} Jul 20 19:28:03.202225 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.202180 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fx9q6" podStartSLOduration=4.41003624 podStartE2EDuration="24.202167224s" totalImagesPullingTime="19.792130984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:03.201836928 +0000 UTC m=+24.826915922" watchObservedRunningTime="2026-07-20 19:28:03.202167224 +0000 UTC m=+24.827246218" Jul 20 19:28:03.227109 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.227051 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zmf8z" podStartSLOduration=4.407762714 podStartE2EDuration="24.227032588s" totalImagesPullingTime="19.819269874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:03.214217615 +0000 UTC m=+24.839296609" watchObservedRunningTime="2026-07-20 19:28:03.227032588 +0000 UTC m=+24.852111585" Jul 20 19:28:03.239100 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.238926 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-48ksk" podStartSLOduration=4.369379487 podStartE2EDuration="24.238907057s" totalImagesPullingTime="19.86952757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:03.238897754 +0000 UTC m=+24.863976749" watchObservedRunningTime="2026-07-20 19:28:03.238907057 +0000 UTC m=+24.863986065" Jul 20 19:28:03.753336 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.753315 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 19:28:03.951083 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.950943 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T19:28:03.753331307Z","UUID":"94ed55d5-8ed7-4108-b477-3d7f66d0e437","Handler":null,"Name":"","Endpoint":""} Jul 20 19:28:03.952294 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.952272 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 19:28:03.952412 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:03.952305 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 19:28:04.055304 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:04.055278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:04.055304 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:04.055290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:04.055555 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:04.055410 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:04.055555 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:04.055530 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:04.184226 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:04.184172 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" event={"ID":"65a53832-7082-4db5-80fb-2df330fb6f2f","Type":"ContainerStarted","Data":"763aac43728eb455acb25e207c8e92719f26a42a54f77fff21bd5933655f01ac"} Jul 20 19:28:04.186166 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:04.185933 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49gfp" event={"ID":"1536be0d-fe2e-4ee0-b8d5-71437163ac9d","Type":"ContainerStarted","Data":"85f663c702c031648012a77839e63aea156b27fa39f6fc8786e8d2dd28990334"} Jul 20 19:28:04.188070 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:04.188044 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" event={"ID":"2f1c6459085a5c415e46ff01a63ff4c0","Type":"ContainerStarted","Data":"1cc48cbc4a01f94229b0f0247cbed5632c82deb6dab5011d34efac6190002b81"} Jul 20 19:28:04.211034 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:04.210937 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-49gfp" podStartSLOduration=5.313959543 podStartE2EDuration="25.210916345s" totalImagesPullingTime="19.896956802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:04.198699842 +0000 UTC m=+25.823778836" watchObservedRunningTime="2026-07-20 19:28:04.210916345 +0000 UTC m=+25.835995341" Jul 20 19:28:04.211246 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:04.211203 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-219.ec2.internal" podStartSLOduration=24.211187109 podStartE2EDuration="24.211187109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:04.210639851 +0000 UTC m=+25.835718846" watchObservedRunningTime="2026-07-20 19:28:04.211187109 +0000 UTC m=+25.836266101" Jul 20 19:28:05.056177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:05.056150 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:05.056329 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:05.056271 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:05.194796 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:05.194758 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"1d6f4f93667613074f30f52a7183cca74f9526636bdf3cfcc5a1940399c3f0aa"} Jul 20 19:28:05.197117 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:05.197087 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" event={"ID":"65a53832-7082-4db5-80fb-2df330fb6f2f","Type":"ContainerStarted","Data":"e90b3c78292b1b7300272ad9a3d3063695c221e977e6c335f5538a028035fcd5"} Jul 20 19:28:05.213034 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:05.212994 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-whv8r" podStartSLOduration=3.7212331279999997 podStartE2EDuration="26.212981158s" totalImagesPullingTime="22.49174803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:05.212640957 +0000 UTC m=+26.837719963" watchObservedRunningTime="2026-07-20 19:28:05.212981158 +0000 UTC m=+26.838060152" Jul 20 19:28:06.056133 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:06.056089 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:06.056300 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:06.056089 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:06.056300 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:06.056210 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:06.056300 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:06.056274 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:06.268683 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:06.268650 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:28:06.269197 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:06.269175 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:28:07.055476 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:07.055385 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:07.055629 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:07.055529 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:07.201036 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:07.201008 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:28:07.201589 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:07.201564 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fx9q6" Jul 20 19:28:08.055980 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:08.055864 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:08.056301 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:08.055925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:08.056301 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:08.056053 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:08.056301 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:08.056183 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:08.206483 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:08.206454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" event={"ID":"b410358b-9644-4059-8c56-a03ab134ebca","Type":"ContainerStarted","Data":"ef59985276062fadef9285ba33f6cc190312babf6e1b240015b74b5bad93926c"} Jul 20 19:28:08.229408 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:08.229365 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" podStartSLOduration=9.253462362 podStartE2EDuration="29.229348395s" totalImagesPullingTime="19.975886033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:08.228121624 +0000 UTC m=+29.853200618" watchObservedRunningTime="2026-07-20 19:28:08.229348395 +0000 UTC m=+29.854427388" Jul 20 19:28:09.056228 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.056198 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:09.056829 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:09.056279 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:09.209613 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.209581 2580 generic.go:356] "Generic (PLEG): container finished" podID="98b76723-379b-49eb-ae59-2d6122a50121" containerID="b016491ebe7143a7045803e077907dd8b05c98a642b40f764cc26c1bcec25748" exitCode=0 Jul 20 19:28:09.209769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.209668 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerDied","Data":"b016491ebe7143a7045803e077907dd8b05c98a642b40f764cc26c1bcec25748"} Jul 20 19:28:09.209941 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.209917 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:28:09.210041 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.210004 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:28:09.213916 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.211875 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:28:09.226128 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.226107 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:28:09.226316 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:09.226301 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:28:10.056190 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.056024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:10.056380 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.056079 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:10.056380 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:10.056265 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:10.056380 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:10.056348 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:10.167837 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.167807 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vs4fj"] Jul 20 19:28:10.167976 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.167914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:10.168021 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:10.167988 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:10.170616 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.170589 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b76ts"] Jul 20 19:28:10.181504 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.181407 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dgq65"] Jul 20 19:28:10.211604 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.211584 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:10.211726 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:10.211607 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:10.211726 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:10.211700 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:10.212150 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:10.212126 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:11.216580 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:11.216516 2580 generic.go:356] "Generic (PLEG): container finished" podID="98b76723-379b-49eb-ae59-2d6122a50121" containerID="e881b9cfc4104b3a4c2dc083808b463480b40ba987e8e25feac8af149b1496e6" exitCode=0 Jul 20 19:28:11.217041 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:11.216591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerDied","Data":"e881b9cfc4104b3a4c2dc083808b463480b40ba987e8e25feac8af149b1496e6"} Jul 20 19:28:12.055944 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:12.055909 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:12.056084 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:12.056005 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:12.056084 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.056014 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:12.056084 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:12.056035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:12.056196 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.056109 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:12.056196 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.056183 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:12.499438 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:12.499402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:12.499779 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:12.499465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:12.499779 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.499558 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:28:12.499779 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.499574 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:28:12.499779 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.499633 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret podName:c1e1ab83-94d8-4a81-9d3f-c14b59e8e542 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:44.499612152 +0000 UTC m=+66.124691142 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret") pod "global-pull-secret-syncer-dgq65" (UID: "c1e1ab83-94d8-4a81-9d3f-c14b59e8e542") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:28:12.499779 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.499683 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:44.499646073 +0000 UTC m=+66.124725082 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:28:12.600611 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:12.600554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:12.600722 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.600689 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:28:12.600722 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.600704 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:28:12.600722 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.600713 2580 projected.go:196] Error preparing data for projected volume kube-api-access-pdq4x for pod openshift-network-diagnostics/network-check-target-vs4fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:28:12.600813 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:12.600759 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x podName:03d18b33-38dc-49c5-9c9b-ef12ac983df3 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:44.600742 +0000 UTC m=+66.225820974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pdq4x" (UniqueName: "kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x") pod "network-check-target-vs4fj" (UID: "03d18b33-38dc-49c5-9c9b-ef12ac983df3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:28:13.222658 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:13.222577 2580 generic.go:356] "Generic (PLEG): container finished" podID="98b76723-379b-49eb-ae59-2d6122a50121" containerID="647d15158ddbacb840f3b70140074ba4eb6b9e618912a04354e8f89f0db3b151" exitCode=0 Jul 20 19:28:13.222658 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:13.222625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerDied","Data":"647d15158ddbacb840f3b70140074ba4eb6b9e618912a04354e8f89f0db3b151"} Jul 20 19:28:14.056024 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.055975 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:14.056024 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.056010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:14.056535 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.056010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:14.056535 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:14.056108 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vs4fj" podUID="03d18b33-38dc-49c5-9c9b-ef12ac983df3" Jul 20 19:28:14.056535 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:14.056241 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:28:14.056535 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:14.056319 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgq65" podUID="c1e1ab83-94d8-4a81-9d3f-c14b59e8e542" Jul 20 19:28:14.702332 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.702262 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-219.ec2.internal" event="NodeReady" Jul 20 19:28:14.702506 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.702407 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 19:28:14.752042 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.751977 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tvwsq"] Jul 20 19:28:14.754216 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.754169 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qmn6p"] Jul 20 19:28:14.754355 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.754322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.756212 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.756182 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:14.756948 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.756924 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 19:28:14.757055 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.756933 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 19:28:14.757055 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.757043 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27sg8\"" Jul 20 19:28:14.758755 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.758732 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 19:28:14.759198 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.759156 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 19:28:14.759635 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.759616 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 19:28:14.759750 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.759673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-94xxz\"" Jul 20 19:28:14.764743 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.764722 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qmn6p"] Jul 20 19:28:14.767999 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.767950 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tvwsq"] Jul 20 19:28:14.819102 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.819062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4glln\" (UniqueName: \"kubernetes.io/projected/c967fb7b-6a3f-428a-8600-a7c39c1973af-kube-api-access-4glln\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.819264 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.819126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c967fb7b-6a3f-428a-8600-a7c39c1973af-tmp-dir\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.819264 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.819173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c967fb7b-6a3f-428a-8600-a7c39c1973af-config-volume\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.819264 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.819244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.920416 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.920381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c967fb7b-6a3f-428a-8600-a7c39c1973af-tmp-dir\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.920578 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.920444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c967fb7b-6a3f-428a-8600-a7c39c1973af-config-volume\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.920578 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.920483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chnwc\" (UniqueName: \"kubernetes.io/projected/ce9b176e-f8c6-45de-bd20-fc9f8b9cf013-kube-api-access-chnwc\") pod \"ingress-canary-qmn6p\" (UID: \"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013\") " pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:14.920578 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.920555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.920687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.920577 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce9b176e-f8c6-45de-bd20-fc9f8b9cf013-cert\") pod \"ingress-canary-qmn6p\" (UID: \"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013\") " pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:14.920687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.920610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4glln\" (UniqueName: \"kubernetes.io/projected/c967fb7b-6a3f-428a-8600-a7c39c1973af-kube-api-access-4glln\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.920687 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:14.920676 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:14.920832 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:14.920814 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:28:15.420790405 +0000 UTC m=+37.045869381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:28:14.921006 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.920992 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c967fb7b-6a3f-428a-8600-a7c39c1973af-tmp-dir\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.921225 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.921208 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c967fb7b-6a3f-428a-8600-a7c39c1973af-config-volume\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:14.933403 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:14.933255 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4glln\" (UniqueName: \"kubernetes.io/projected/c967fb7b-6a3f-428a-8600-a7c39c1973af-kube-api-access-4glln\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:15.021813 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:15.021737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chnwc\" (UniqueName: \"kubernetes.io/projected/ce9b176e-f8c6-45de-bd20-fc9f8b9cf013-kube-api-access-chnwc\") pod \"ingress-canary-qmn6p\" (UID: \"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013\") " pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:15.021813 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:15.021793 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce9b176e-f8c6-45de-bd20-fc9f8b9cf013-cert\") pod \"ingress-canary-qmn6p\" (UID: \"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013\") " pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:15.024687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:15.024665 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce9b176e-f8c6-45de-bd20-fc9f8b9cf013-cert\") pod \"ingress-canary-qmn6p\" (UID: \"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013\") " pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:15.030323 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:15.030299 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chnwc\" (UniqueName: \"kubernetes.io/projected/ce9b176e-f8c6-45de-bd20-fc9f8b9cf013-kube-api-access-chnwc\") pod \"ingress-canary-qmn6p\" (UID: \"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013\") " pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:15.074474 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:15.074452 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qmn6p" Jul 20 19:28:15.245059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:15.245017 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qmn6p"] Jul 20 19:28:15.249906 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:28:15.249876 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce9b176e_f8c6_45de_bd20_fc9f8b9cf013.slice/crio-e996b47f30a38b2748ce2021b644b9e70ce4eb1790d0b9c01c0f4e9add3da4cd WatchSource:0}: Error finding container e996b47f30a38b2748ce2021b644b9e70ce4eb1790d0b9c01c0f4e9add3da4cd: Status 404 returned error can't find the container with id e996b47f30a38b2748ce2021b644b9e70ce4eb1790d0b9c01c0f4e9add3da4cd Jul 20 19:28:15.424707 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:15.424674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:15.424864 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:15.424781 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:15.424864 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:15.424832 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:28:16.42481885 +0000 UTC m=+38.049897822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:28:16.055668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.055601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:16.055668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.055656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:16.055918 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.055726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:16.057845 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.057813 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 19:28:16.058067 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.058036 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 19:28:16.058067 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.058060 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pw7wf\"" Jul 20 19:28:16.058335 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.058315 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 19:28:16.058437 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.058391 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqcxh\"" Jul 20 19:28:16.058500 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.058454 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 19:28:16.232422 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.232386 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qmn6p" event={"ID":"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013","Type":"ContainerStarted","Data":"e996b47f30a38b2748ce2021b644b9e70ce4eb1790d0b9c01c0f4e9add3da4cd"} Jul 20 19:28:16.434018 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:16.433939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:16.434188 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:16.434080 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:16.434188 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:16.434155 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:28:18.434133114 +0000 UTC m=+40.059212104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:28:18.449940 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:18.449911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:18.450266 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:18.450061 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:18.450266 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:18.450122 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:28:22.450106308 +0000 UTC m=+44.075185280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:28:19.241087 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:19.241050 2580 generic.go:356] "Generic (PLEG): container finished" podID="98b76723-379b-49eb-ae59-2d6122a50121" containerID="fad7cef02715585f973e31a97862085322927060e6b013f9553ed643a8f28793" exitCode=0 Jul 20 19:28:19.241267 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:19.241117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerDied","Data":"fad7cef02715585f973e31a97862085322927060e6b013f9553ed643a8f28793"} Jul 20 19:28:19.245935 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:19.245774 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qmn6p" event={"ID":"ce9b176e-f8c6-45de-bd20-fc9f8b9cf013","Type":"ContainerStarted","Data":"8747d715480b58d929a2f65ef447dc31b4ded4407167b74fe9551e809561ab2d"} Jul 20 19:28:19.278390 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:19.278344 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qmn6p" podStartSLOduration=2.228942563 podStartE2EDuration="5.278328226s" totalImagesPullingTime="3.049385663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:19.277546233 +0000 UTC m=+40.902625228" watchObservedRunningTime="2026-07-20 19:28:19.278328226 +0000 UTC m=+40.903407221" Jul 20 19:28:20.250827 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:20.250789 2580 generic.go:356] "Generic (PLEG): container finished" podID="98b76723-379b-49eb-ae59-2d6122a50121" containerID="010d32ab7255e3a50e30db2cea1bc1fe9b3d30dab50dd8926bb4d6c28c0fdfe1" exitCode=0 Jul 20 19:28:20.251227 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:20.250871 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerDied","Data":"010d32ab7255e3a50e30db2cea1bc1fe9b3d30dab50dd8926bb4d6c28c0fdfe1"} Jul 20 19:28:21.256563 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:21.256524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" event={"ID":"98b76723-379b-49eb-ae59-2d6122a50121","Type":"ContainerStarted","Data":"c0b15315d3ec5b36b30913e852127ab64ba4402fa1e410b0e0d39de8b4cc5dd6"} Jul 20 19:28:21.277958 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:21.277917 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8f4nr" podStartSLOduration=9.68967539 podStartE2EDuration="42.277902779s" totalImagesPullingTime="32.588227389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:21.27729231 +0000 UTC m=+42.902371303" watchObservedRunningTime="2026-07-20 19:28:21.277902779 +0000 UTC m=+42.902981773" Jul 20 19:28:22.476738 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:22.476698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:22.477086 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:22.476853 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:22.477086 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:22.476927 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:28:30.476902355 +0000 UTC m=+52.101981327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:28:30.532302 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:30.532262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:30.532773 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:30.532378 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:30.532773 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:30.532457 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:28:46.532438892 +0000 UTC m=+68.157517877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:28:41.239766 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:41.239730 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2zn5" Jul 20 19:28:44.530706 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.530650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:44.530706 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.530712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:28:44.532902 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.532884 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 19:28:44.532950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.532906 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 19:28:44.541435 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:44.541412 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 19:28:44.541518 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:44.541506 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:29:48.541487004 +0000 UTC m=+130.166565995 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : secret "metrics-daemon-secret" not found Jul 20 19:28:44.544667 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.544649 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1e1ab83-94d8-4a81-9d3f-c14b59e8e542-original-pull-secret\") pod \"global-pull-secret-syncer-dgq65\" (UID: \"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542\") " pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:44.569688 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.569667 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgq65" Jul 20 19:28:44.631115 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.631076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:44.633759 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.633738 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 19:28:44.644264 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.644243 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 19:28:44.653860 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.653835 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdq4x\" (UniqueName: \"kubernetes.io/projected/03d18b33-38dc-49c5-9c9b-ef12ac983df3-kube-api-access-pdq4x\") pod \"network-check-target-vs4fj\" (UID: \"03d18b33-38dc-49c5-9c9b-ef12ac983df3\") " pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:44.695982 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.695955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dgq65"] Jul 20 19:28:44.699180 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:28:44.699154 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e1ab83_94d8_4a81_9d3f_c14b59e8e542.slice/crio-8792ddcc79f71bb98382be3896522e74c151cf3d2a56b5a98c007a24b11af5ec WatchSource:0}: Error finding container 8792ddcc79f71bb98382be3896522e74c151cf3d2a56b5a98c007a24b11af5ec: Status 404 returned error can't find the container with id 8792ddcc79f71bb98382be3896522e74c151cf3d2a56b5a98c007a24b11af5ec Jul 20 19:28:44.886052 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.886024 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqcxh\"" Jul 20 19:28:44.895050 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:44.895030 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:45.009560 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:45.009529 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vs4fj"] Jul 20 19:28:45.012492 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:28:45.012463 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d18b33_38dc_49c5_9c9b_ef12ac983df3.slice/crio-4215680a10b8c8da91761cdf59ea020d5f0a89dd0bdb0ad1993f104c764d9f4c WatchSource:0}: Error finding container 4215680a10b8c8da91761cdf59ea020d5f0a89dd0bdb0ad1993f104c764d9f4c: Status 404 returned error can't find the container with id 4215680a10b8c8da91761cdf59ea020d5f0a89dd0bdb0ad1993f104c764d9f4c Jul 20 19:28:45.314114 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:45.314074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vs4fj" event={"ID":"03d18b33-38dc-49c5-9c9b-ef12ac983df3","Type":"ContainerStarted","Data":"4215680a10b8c8da91761cdf59ea020d5f0a89dd0bdb0ad1993f104c764d9f4c"} Jul 20 19:28:45.315058 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:45.315033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dgq65" event={"ID":"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542","Type":"ContainerStarted","Data":"8792ddcc79f71bb98382be3896522e74c151cf3d2a56b5a98c007a24b11af5ec"} Jul 20 19:28:46.543892 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:46.543801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:28:46.544358 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:46.544052 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:46.544358 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:28:46.544137 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:29:18.544116452 +0000 UTC m=+100.169195426 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:28:50.329233 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:50.329137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vs4fj" event={"ID":"03d18b33-38dc-49c5-9c9b-ef12ac983df3","Type":"ContainerStarted","Data":"0311a9e6442e5891a7cab628e83c99b88bea0de097179226a62e22281ec9a07b"} Jul 20 19:28:50.329721 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:50.329246 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:28:50.332867 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:50.332836 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dgq65" event={"ID":"c1e1ab83-94d8-4a81-9d3f-c14b59e8e542","Type":"ContainerStarted","Data":"8612da578651091cd383c51d49e9a4d58a4e723ec48175e4156bce8eb8263d19"} Jul 20 19:28:50.344704 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:50.344651 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vs4fj" podStartSLOduration=66.259540665 podStartE2EDuration="1m11.344635476s" totalImagesPullingTime="5.085094811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:50.344171168 +0000 UTC m=+71.969250162" watchObservedRunningTime="2026-07-20 19:28:50.344635476 +0000 UTC m=+71.969714472" Jul 20 19:28:50.357580 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:28:50.357529 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dgq65" podStartSLOduration=65.96905267 podStartE2EDuration="1m11.35751123s" totalImagesPullingTime="5.38845856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:50.356889455 +0000 UTC m=+71.981968470" watchObservedRunningTime="2026-07-20 19:28:50.35751123 +0000 UTC m=+71.982590225" Jul 20 19:29:05.142539 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.142414 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp"] Jul 20 19:29:05.144678 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.144662 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.148766 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.147446 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w66dh\"" Jul 20 19:29:05.148766 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.147866 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 20 19:29:05.148766 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.148296 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 20 19:29:05.148766 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.148570 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 20 19:29:05.149237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.149216 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 20 19:29:05.162042 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.162018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp"] Jul 20 19:29:05.168136 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.168116 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53d36a8a-e1d3-40be-baef-97d2b5a62d39-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-746454f4d-g5mvp\" (UID: \"53d36a8a-e1d3-40be-baef-97d2b5a62d39\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.168229 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.168149 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzxqn\" (UniqueName: \"kubernetes.io/projected/53d36a8a-e1d3-40be-baef-97d2b5a62d39-kube-api-access-lzxqn\") pod \"managed-serviceaccount-addon-agent-746454f4d-g5mvp\" (UID: \"53d36a8a-e1d3-40be-baef-97d2b5a62d39\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.268698 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.268658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53d36a8a-e1d3-40be-baef-97d2b5a62d39-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-746454f4d-g5mvp\" (UID: \"53d36a8a-e1d3-40be-baef-97d2b5a62d39\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.268698 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.268704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzxqn\" (UniqueName: \"kubernetes.io/projected/53d36a8a-e1d3-40be-baef-97d2b5a62d39-kube-api-access-lzxqn\") pod \"managed-serviceaccount-addon-agent-746454f4d-g5mvp\" (UID: \"53d36a8a-e1d3-40be-baef-97d2b5a62d39\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.271095 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.271071 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53d36a8a-e1d3-40be-baef-97d2b5a62d39-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-746454f4d-g5mvp\" (UID: \"53d36a8a-e1d3-40be-baef-97d2b5a62d39\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.278186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.278161 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzxqn\" (UniqueName: \"kubernetes.io/projected/53d36a8a-e1d3-40be-baef-97d2b5a62d39-kube-api-access-lzxqn\") pod \"managed-serviceaccount-addon-agent-746454f4d-g5mvp\" (UID: \"53d36a8a-e1d3-40be-baef-97d2b5a62d39\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.468247 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.468146 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" Jul 20 19:29:05.583350 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:05.583306 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp"] Jul 20 19:29:05.586971 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:29:05.586942 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53d36a8a_e1d3_40be_baef_97d2b5a62d39.slice/crio-6181a56d0348bf800352e736648b069a83411f82ca04c1e6a53f91dd32f73ab6 WatchSource:0}: Error finding container 6181a56d0348bf800352e736648b069a83411f82ca04c1e6a53f91dd32f73ab6: Status 404 returned error can't find the container with id 6181a56d0348bf800352e736648b069a83411f82ca04c1e6a53f91dd32f73ab6 Jul 20 19:29:06.371761 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:06.371726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" event={"ID":"53d36a8a-e1d3-40be-baef-97d2b5a62d39","Type":"ContainerStarted","Data":"6181a56d0348bf800352e736648b069a83411f82ca04c1e6a53f91dd32f73ab6"} Jul 20 19:29:08.378275 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:08.378186 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" event={"ID":"53d36a8a-e1d3-40be-baef-97d2b5a62d39","Type":"ContainerStarted","Data":"b3110ee9ff83ef02aa444fdbe3823cddd0e4aba41f00e474757585860bd25816"} Jul 20 19:29:08.391910 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:08.391867 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" podStartSLOduration=0.93919916 podStartE2EDuration="3.391853146s" totalImagesPullingTime="2.452653986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:29:08.391407986 +0000 UTC m=+90.016486980" watchObservedRunningTime="2026-07-20 19:29:08.391853146 +0000 UTC m=+90.016932163" Jul 20 19:29:18.566534 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:18.566474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:29:18.567099 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:29:18.566623 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:29:18.567099 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:29:18.566697 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:30:22.566680649 +0000 UTC m=+164.191759621 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:29:21.337504 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:21.337472 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vs4fj" Jul 20 19:29:48.563063 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:29:48.563000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:29:48.563576 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:29:48.563149 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 19:29:48.563576 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:29:48.563229 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:50.563205152 +0000 UTC m=+252.188284137 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : secret "metrics-daemon-secret" not found Jul 20 19:30:17.767831 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:30:17.767779 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tvwsq" podUID="c967fb7b-6a3f-428a-8600-a7c39c1973af" Jul 20 19:30:18.536583 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:30:18.536547 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tvwsq" Jul 20 19:30:19.078563 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:30:19.078529 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:30:22.585201 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:30:22.585157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:30:22.585594 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:30:22.585293 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:30:22.585594 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:30:22.585363 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls podName:c967fb7b-6a3f-428a-8600-a7c39c1973af nodeName:}" failed. No retries permitted until 2026-07-20 19:32:24.585345754 +0000 UTC m=+286.210424745 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls") pod "dns-default-tvwsq" (UID: "c967fb7b-6a3f-428a-8600-a7c39c1973af") : secret "dns-default-metrics-tls" not found Jul 20 19:30:33.055790 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:30:33.055709 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:31:08.652818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:08.652782 2580 generic.go:356] "Generic (PLEG): container finished" podID="53d36a8a-e1d3-40be-baef-97d2b5a62d39" containerID="b3110ee9ff83ef02aa444fdbe3823cddd0e4aba41f00e474757585860bd25816" exitCode=255 Jul 20 19:31:08.653252 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:08.652823 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" event={"ID":"53d36a8a-e1d3-40be-baef-97d2b5a62d39","Type":"ContainerDied","Data":"b3110ee9ff83ef02aa444fdbe3823cddd0e4aba41f00e474757585860bd25816"} Jul 20 19:31:08.658297 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:08.658277 2580 scope.go:117] "RemoveContainer" containerID="b3110ee9ff83ef02aa444fdbe3823cddd0e4aba41f00e474757585860bd25816" Jul 20 19:31:09.656597 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:09.656558 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-746454f4d-g5mvp" event={"ID":"53d36a8a-e1d3-40be-baef-97d2b5a62d39","Type":"ContainerStarted","Data":"c73c8252b2eedcf18d2715f38fd5230e7a7d808d13364787109c49c53aee886f"} Jul 20 19:31:39.035115 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.035075 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79d667788f-h69c8"] Jul 20 19:31:39.038271 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.038249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.040075 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.040054 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dpg8d\"" Jul 20 19:31:39.040075 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.040072 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 19:31:39.042536 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.042508 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 19:31:39.043259 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.043239 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 19:31:39.050178 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.050159 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 19:31:39.062087 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.062060 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79d667788f-h69c8"] Jul 20 19:31:39.099766 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.099718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99a905de-4b29-429d-b472-41d5c51557b2-ca-trust-extracted\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.099943 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.099789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz7ls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-kube-api-access-cz7ls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.099943 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.099889 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-image-registry-private-configuration\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.100107 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.100083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-registry-certificates\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.100239 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.100221 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-installation-pull-secrets\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.100338 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.100323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-bound-sa-token\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.100469 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.100455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.100608 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.100593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-trusted-ca\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.143229 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.143189 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59844479fb-ld75n"] Jul 20 19:31:39.146256 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.146239 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.151845 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.151811 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 20 19:31:39.152093 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.152061 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 20 19:31:39.152460 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.152148 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-stats-default\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" Jul 20 19:31:39.152460 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.152181 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-dockercfg-hqm6g\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hqm6g\"" type="*v1.Secret" Jul 20 19:31:39.152460 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.152201 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"router-default-59844479fb-ld75n\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="100f1256-c5f4-4a3b-bc90-daba6644692c" pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.152460 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.152207 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-metrics-certs-default\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" Jul 20 19:31:39.152460 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.152289 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-ingress-cert\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" Jul 20 19:31:39.153016 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.152997 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k"] Jul 20 19:31:39.156006 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.155991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" Jul 20 19:31:39.161053 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.161026 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc"] Jul 20 19:31:39.163752 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.163732 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-c9c7t\"" Jul 20 19:31:39.163948 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.163934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.166951 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.166932 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 20 19:31:39.168733 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.168710 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 20 19:31:39.168838 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.168769 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:31:39.168950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.168922 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 20 19:31:39.169176 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.169068 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wnk7c\"" Jul 20 19:31:39.169500 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.169304 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 20 19:31:39.170156 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.170139 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-qqmfb"] Jul 20 19:31:39.172859 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.172844 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.180963 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.180940 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 19:31:39.181759 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.181741 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k"] Jul 20 19:31:39.183365 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.183346 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 19:31:39.183480 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.183370 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 20 19:31:39.183480 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.183371 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9nc2g\"" Jul 20 19:31:39.183595 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.183509 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 20 19:31:39.194940 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.194916 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 20 19:31:39.197980 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.197955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc"] Jul 20 19:31:39.199548 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.199525 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59844479fb-ld75n"] Jul 20 19:31:39.201850 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.201827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-image-registry-private-configuration\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.201939 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.201860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-registry-certificates\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.201939 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.201888 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef772c49-d191-4c78-b4fe-13512691a076-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.202034 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-installation-pull-secrets\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.202074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-bound-sa-token\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.202110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbrm7\" (UniqueName: \"kubernetes.io/projected/ef772c49-d191-4c78-b4fe-13512691a076-kube-api-access-cbrm7\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.202153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.202153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmbms\" (UniqueName: \"kubernetes.io/projected/100f1256-c5f4-4a3b-bc90-daba6644692c-kube-api-access-kmbms\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.202227 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202176 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-stats-auth\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.202227 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8ggs\" (UniqueName: \"kubernetes.io/projected/b8e86573-7b3f-4609-83ca-b7c672fa3b58-kube-api-access-l8ggs\") pod \"network-check-source-7bc5b6bf85-xjg6k\" (UID: \"b8e86573-7b3f-4609-83ca-b7c672fa3b58\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" Jul 20 19:31:39.202297 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-trusted-ca\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.202297 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99a905de-4b29-429d-b472-41d5c51557b2-ca-trust-extracted\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.202391 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz7ls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-kube-api-access-cz7ls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.202458 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-default-certificate\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.202458 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202414 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.202562 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef772c49-d191-4c78-b4fe-13512691a076-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.202562 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.202661 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.202569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-registry-certificates\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.202827 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.202804 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:31:39.203002 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.202980 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d667788f-h69c8: secret "image-registry-tls" not found Jul 20 19:31:39.203390 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.203375 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls podName:99a905de-4b29-429d-b472-41d5c51557b2 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:39.703353049 +0000 UTC m=+241.328432037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls") pod "image-registry-79d667788f-h69c8" (UID: "99a905de-4b29-429d-b472-41d5c51557b2") : secret "image-registry-tls" not found Jul 20 19:31:39.203716 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.203249 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99a905de-4b29-429d-b472-41d5c51557b2-ca-trust-extracted\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.203892 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.203867 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-trusted-ca\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.205071 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.205047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-image-registry-private-configuration\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.205186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.205047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-installation-pull-secrets\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.205456 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.205382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-qqmfb"] Jul 20 19:31:39.217381 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.217290 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-bound-sa-token\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.217648 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.217600 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz7ls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-kube-api-access-cz7ls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.303415 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-default-certificate\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.303415 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef772c49-d191-4c78-b4fe-13512691a076-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.303415 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.303415 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqg74\" (UniqueName: \"kubernetes.io/projected/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-kube-api-access-xqg74\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.303769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-service-ca-bundle\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.303769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.303769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-snapshots\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.303769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.303769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmbms\" (UniqueName: \"kubernetes.io/projected/100f1256-c5f4-4a3b-bc90-daba6644692c-kube-api-access-kmbms\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.303769 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.303743 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:39.803722801 +0000 UTC m=+241.428801799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : configmap references non-existent config key: service-ca.crt Jul 20 19:31:39.304110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbrm7\" (UniqueName: \"kubernetes.io/projected/ef772c49-d191-4c78-b4fe-13512691a076-kube-api-access-cbrm7\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.304110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303802 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef772c49-d191-4c78-b4fe-13512691a076-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.304110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-serving-cert\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.304110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-tmp\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.304110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-stats-auth\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.304110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.303939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8ggs\" (UniqueName: \"kubernetes.io/projected/b8e86573-7b3f-4609-83ca-b7c672fa3b58-kube-api-access-l8ggs\") pod \"network-check-source-7bc5b6bf85-xjg6k\" (UID: \"b8e86573-7b3f-4609-83ca-b7c672fa3b58\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" Jul 20 19:31:39.304110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.304054 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef772c49-d191-4c78-b4fe-13512691a076-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.305939 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.305923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef772c49-d191-4c78-b4fe-13512691a076-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.311265 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.311241 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8ggs\" (UniqueName: \"kubernetes.io/projected/b8e86573-7b3f-4609-83ca-b7c672fa3b58-kube-api-access-l8ggs\") pod \"network-check-source-7bc5b6bf85-xjg6k\" (UID: \"b8e86573-7b3f-4609-83ca-b7c672fa3b58\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" Jul 20 19:31:39.312174 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.312153 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbrm7\" (UniqueName: \"kubernetes.io/projected/ef772c49-d191-4c78-b4fe-13512691a076-kube-api-access-cbrm7\") pod \"kube-storage-version-migrator-operator-7658d9b749-75wvc\" (UID: \"ef772c49-d191-4c78-b4fe-13512691a076\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.404257 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.404227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-serving-cert\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.404257 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.404259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-tmp\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.404524 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.404367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.404524 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.404490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqg74\" (UniqueName: \"kubernetes.io/projected/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-kube-api-access-xqg74\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.404524 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.404522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-service-ca-bundle\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.404684 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.404551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-snapshots\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.404934 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.404912 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-tmp\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.405144 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.405124 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-snapshots\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.405144 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.405133 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-service-ca-bundle\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.405346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.405329 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.406581 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.406562 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-serving-cert\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.412466 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.412445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqg74\" (UniqueName: \"kubernetes.io/projected/ec1ccec2-98b0-435d-9e8e-c2de5706aadd-kube-api-access-xqg74\") pod \"insights-operator-555d9d7885-qqmfb\" (UID: \"ec1ccec2-98b0-435d-9e8e-c2de5706aadd\") " pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.464050 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.464004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" Jul 20 19:31:39.471616 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.471582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" Jul 20 19:31:39.481546 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.481523 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-qqmfb" Jul 20 19:31:39.613691 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.612380 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k"] Jul 20 19:31:39.615251 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:31:39.615224 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e86573_7b3f_4609_83ca_b7c672fa3b58.slice/crio-09e27592c131b5520311605caaa2a6fe5d67198bf3d3ec72e0bf950181c5e5ad WatchSource:0}: Error finding container 09e27592c131b5520311605caaa2a6fe5d67198bf3d3ec72e0bf950181c5e5ad: Status 404 returned error can't find the container with id 09e27592c131b5520311605caaa2a6fe5d67198bf3d3ec72e0bf950181c5e5ad Jul 20 19:31:39.629994 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.629970 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc"] Jul 20 19:31:39.632490 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:31:39.632459 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef772c49_d191_4c78_b4fe_13512691a076.slice/crio-c400608364c947bf5cdb66f6c1af9bf8d79900f58c9b02ac5613da3417f97d31 WatchSource:0}: Error finding container c400608364c947bf5cdb66f6c1af9bf8d79900f58c9b02ac5613da3417f97d31: Status 404 returned error can't find the container with id c400608364c947bf5cdb66f6c1af9bf8d79900f58c9b02ac5613da3417f97d31 Jul 20 19:31:39.643878 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.643845 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-qqmfb"] Jul 20 19:31:39.647229 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:31:39.647205 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec1ccec2_98b0_435d_9e8e_c2de5706aadd.slice/crio-da00bb5add641669681e340ce1b42243e8048a5679d05bfdcd8e3c0345812a19 WatchSource:0}: Error finding container da00bb5add641669681e340ce1b42243e8048a5679d05bfdcd8e3c0345812a19: Status 404 returned error can't find the container with id da00bb5add641669681e340ce1b42243e8048a5679d05bfdcd8e3c0345812a19 Jul 20 19:31:39.708022 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.707988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:39.708162 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.708146 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:31:39.708216 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.708166 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d667788f-h69c8: secret "image-registry-tls" not found Jul 20 19:31:39.708251 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.708222 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls podName:99a905de-4b29-429d-b472-41d5c51557b2 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:40.708205064 +0000 UTC m=+242.333284054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls") pod "image-registry-79d667788f-h69c8" (UID: "99a905de-4b29-429d-b472-41d5c51557b2") : secret "image-registry-tls" not found Jul 20 19:31:39.731741 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.731699 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qqmfb" event={"ID":"ec1ccec2-98b0-435d-9e8e-c2de5706aadd","Type":"ContainerStarted","Data":"da00bb5add641669681e340ce1b42243e8048a5679d05bfdcd8e3c0345812a19"} Jul 20 19:31:39.732817 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.732786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" event={"ID":"ef772c49-d191-4c78-b4fe-13512691a076","Type":"ContainerStarted","Data":"c400608364c947bf5cdb66f6c1af9bf8d79900f58c9b02ac5613da3417f97d31"} Jul 20 19:31:39.734119 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.734099 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" event={"ID":"b8e86573-7b3f-4609-83ca-b7c672fa3b58","Type":"ContainerStarted","Data":"7c76c3df8b81ad8c9d73cd73ef73441ad1af9432a0f2a0b376ca0c6420465c60"} Jul 20 19:31:39.734197 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.734132 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" event={"ID":"b8e86573-7b3f-4609-83ca-b7c672fa3b58","Type":"ContainerStarted","Data":"09e27592c131b5520311605caaa2a6fe5d67198bf3d3ec72e0bf950181c5e5ad"} Jul 20 19:31:39.749022 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.748968 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-xjg6k" podStartSLOduration=0.748949474 podStartE2EDuration="748.949474ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:39.748045302 +0000 UTC m=+241.373124296" watchObservedRunningTime="2026-07-20 19:31:39.748949474 +0000 UTC m=+241.374028469" Jul 20 19:31:39.808805 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:39.808760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:39.808984 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:39.808902 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:40.808887249 +0000 UTC m=+242.433966222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : configmap references non-existent config key: service-ca.crt Jul 20 19:31:40.104307 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.104272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 20 19:31:40.114311 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.114285 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 19:31:40.114443 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.114380 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:40.614357443 +0000 UTC m=+242.239436432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : secret "router-metrics-certs-default" not found Jul 20 19:31:40.304334 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.304292 2580 secret.go:189] Couldn't get secret openshift-ingress/default-ingress-cert: failed to sync secret cache: timed out waiting for the condition Jul 20 19:31:40.304536 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.304301 2580 secret.go:189] Couldn't get secret openshift-ingress/router-stats-default: failed to sync secret cache: timed out waiting for the condition Jul 20 19:31:40.304536 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.304409 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-default-certificate podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:40.804386902 +0000 UTC m=+242.429465879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "default-certificate" (UniqueName: "kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-default-certificate") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : failed to sync secret cache: timed out waiting for the condition Jul 20 19:31:40.304536 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.304491 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-stats-auth podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:40.804473268 +0000 UTC m=+242.429552249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "stats-auth" (UniqueName: "kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-stats-auth") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : failed to sync secret cache: timed out waiting for the condition Jul 20 19:31:40.308663 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.308633 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 19:31:40.323730 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.323702 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 19:31:40.333868 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.333836 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmbms\" (UniqueName: \"kubernetes.io/projected/100f1256-c5f4-4a3b-bc90-daba6644692c-kube-api-access-kmbms\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:40.381756 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.381673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 20 19:31:40.458783 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.458750 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hqm6g\"" Jul 20 19:31:40.616704 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.616666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:40.616937 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.616916 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 19:31:40.617031 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.617007 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:41.616985014 +0000 UTC m=+243.242063987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : secret "router-metrics-certs-default" not found Jul 20 19:31:40.718120 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.718023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:40.718283 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.718183 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:31:40.718283 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.718198 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d667788f-h69c8: secret "image-registry-tls" not found Jul 20 19:31:40.718283 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.718257 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls podName:99a905de-4b29-429d-b472-41d5c51557b2 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:42.718237308 +0000 UTC m=+244.343316283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls") pod "image-registry-79d667788f-h69c8" (UID: "99a905de-4b29-429d-b472-41d5c51557b2") : secret "image-registry-tls" not found Jul 20 19:31:40.722720 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.722691 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 20 19:31:40.818865 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.818827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-stats-auth\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:40.819040 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.818891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-default-certificate\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:40.819040 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.818945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:40.819169 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:40.819065 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:42.819051592 +0000 UTC m=+244.444130563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : configmap references non-existent config key: service-ca.crt Jul 20 19:31:40.821375 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.821346 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-stats-auth\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:40.821514 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:40.821382 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-default-certificate\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:41.625710 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:41.625673 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:41.626190 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:41.625810 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 19:31:41.626190 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:41.625876 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:43.625859612 +0000 UTC m=+245.250938585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : secret "router-metrics-certs-default" not found Jul 20 19:31:42.735405 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:42.735306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:42.735918 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:42.735491 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:31:42.735918 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:42.735514 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d667788f-h69c8: secret "image-registry-tls" not found Jul 20 19:31:42.735918 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:42.735569 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls podName:99a905de-4b29-429d-b472-41d5c51557b2 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:46.735552625 +0000 UTC m=+248.360631598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls") pod "image-registry-79d667788f-h69c8" (UID: "99a905de-4b29-429d-b472-41d5c51557b2") : secret "image-registry-tls" not found Jul 20 19:31:42.745532 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:42.745494 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qqmfb" event={"ID":"ec1ccec2-98b0-435d-9e8e-c2de5706aadd","Type":"ContainerStarted","Data":"b5dc65d66f1603bc762d8f80925144c39c0186e1479fce35ea62fa1472b2a048"} Jul 20 19:31:42.747004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:42.746969 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" event={"ID":"ef772c49-d191-4c78-b4fe-13512691a076","Type":"ContainerStarted","Data":"81e8c5104e82c1f76b167f95f93a2abce9a389c46f1aa715adabd9c284649edf"} Jul 20 19:31:42.760761 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:42.760704 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-qqmfb" podStartSLOduration=1.016123411 podStartE2EDuration="3.76068462s" totalImagesPullingTime="2.744561209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:42.760350726 +0000 UTC m=+244.385429721" watchObservedRunningTime="2026-07-20 19:31:42.76068462 +0000 UTC m=+244.385763615" Jul 20 19:31:42.775758 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:42.775703 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" podStartSLOduration=1.018358456 podStartE2EDuration="3.775687225s" totalImagesPullingTime="2.757328769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:42.774622022 +0000 UTC m=+244.399701019" watchObservedRunningTime="2026-07-20 19:31:42.775687225 +0000 UTC m=+244.400766262" Jul 20 19:31:42.836211 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:42.836172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:42.836935 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:42.836917 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:46.836894002 +0000 UTC m=+248.461972979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : configmap references non-existent config key: service-ca.crt Jul 20 19:31:43.421278 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.421243 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4rdtp"] Jul 20 19:31:43.424470 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.424449 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.427271 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.427249 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 19:31:43.427386 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.427366 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8lkqn\"" Jul 20 19:31:43.427677 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.427661 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 19:31:43.441978 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.441955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4rdtp"] Jul 20 19:31:43.540711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.540665 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-crio-socket\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.540711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.540710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.540944 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.540735 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.540944 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.540792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-data-volume\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.540944 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.540811 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwq2x\" (UniqueName: \"kubernetes.io/projected/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-api-access-mwq2x\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641249 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-data-volume\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwq2x\" (UniqueName: \"kubernetes.io/projected/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-api-access-mwq2x\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:43.641554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-crio-socket\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641516 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-crio-socket\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641554 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:43.641533 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 19:31:43.641778 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:43.641621 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:47.641598372 +0000 UTC m=+249.266677348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : secret "router-metrics-certs-default" not found Jul 20 19:31:43.641778 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:43.641617 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:43.641778 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.641652 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-data-volume\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:43.641778 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:43.641702 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:44.141684766 +0000 UTC m=+245.766763752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : configmap "kube-rbac-proxy" not found Jul 20 19:31:43.641778 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:43.641745 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:43.641778 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:43.641781 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:44.141769081 +0000 UTC m=+245.766848053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:43.651728 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:43.651701 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwq2x\" (UniqueName: \"kubernetes.io/projected/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-api-access-mwq2x\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:44.146467 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:44.146414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:44.146957 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:44.146600 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:44.146957 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:44.146674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:44.146957 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:44.146686 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:45.14666393 +0000 UTC m=+246.771742904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : configmap "kube-rbac-proxy" not found Jul 20 19:31:44.146957 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:44.146779 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:44.146957 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:44.146826 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:45.146812011 +0000 UTC m=+246.771890983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:45.155308 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:45.155267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:45.155308 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:45.155306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:45.155759 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:45.155378 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:45.155759 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:45.155409 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:45.155759 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:45.155459 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:47.155417583 +0000 UTC m=+248.780496569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : configmap "kube-rbac-proxy" not found Jul 20 19:31:45.155759 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:45.155480 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:47.155471079 +0000 UTC m=+248.780550051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:45.195572 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:45.195541 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-48ksk_40ce65db-b05a-4930-927c-b81a075191df/dns-node-resolver/0.log" Jul 20 19:31:46.443410 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.443375 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-m9fjk"] Jul 20 19:31:46.446322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.446303 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.448365 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.448343 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 20 19:31:46.448774 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.448756 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 20 19:31:46.448868 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.448797 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r8qfn\"" Jul 20 19:31:46.448868 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.448813 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 20 19:31:46.448868 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.448816 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 20 19:31:46.453811 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.453789 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-m9fjk"] Jul 20 19:31:46.565016 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.564979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06a6cfe6-0400-460c-bb7d-9647c88ca50a-signing-key\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.565197 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.565038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06a6cfe6-0400-460c-bb7d-9647c88ca50a-signing-cabundle\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.565197 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.565056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7868f\" (UniqueName: \"kubernetes.io/projected/06a6cfe6-0400-460c-bb7d-9647c88ca50a-kube-api-access-7868f\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.602681 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.602654 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmf8z_5fa9cad5-2584-4972-9ab0-47dd94309bdc/node-ca/0.log" Jul 20 19:31:46.666243 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.666202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06a6cfe6-0400-460c-bb7d-9647c88ca50a-signing-cabundle\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.666243 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.666245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7868f\" (UniqueName: \"kubernetes.io/projected/06a6cfe6-0400-460c-bb7d-9647c88ca50a-kube-api-access-7868f\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.666475 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.666374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06a6cfe6-0400-460c-bb7d-9647c88ca50a-signing-key\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.666920 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.666901 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06a6cfe6-0400-460c-bb7d-9647c88ca50a-signing-cabundle\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.668712 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.668694 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06a6cfe6-0400-460c-bb7d-9647c88ca50a-signing-key\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.674283 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.674257 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7868f\" (UniqueName: \"kubernetes.io/projected/06a6cfe6-0400-460c-bb7d-9647c88ca50a-kube-api-access-7868f\") pod \"service-ca-58598695f7-m9fjk\" (UID: \"06a6cfe6-0400-460c-bb7d-9647c88ca50a\") " pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.755773 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.755680 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-m9fjk" Jul 20 19:31:46.767795 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.767766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:46.767967 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:46.767946 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:31:46.768018 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:46.767972 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79d667788f-h69c8: secret "image-registry-tls" not found Jul 20 19:31:46.768054 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:46.768034 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls podName:99a905de-4b29-429d-b472-41d5c51557b2 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:54.768014333 +0000 UTC m=+256.393093320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls") pod "image-registry-79d667788f-h69c8" (UID: "99a905de-4b29-429d-b472-41d5c51557b2") : secret "image-registry-tls" not found Jul 20 19:31:46.868245 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.868212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:46.868381 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:46.868339 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:54.868325487 +0000 UTC m=+256.493404458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : configmap references non-existent config key: service-ca.crt Jul 20 19:31:46.879299 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:46.879268 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-m9fjk"] Jul 20 19:31:46.881856 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:31:46.881834 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a6cfe6_0400_460c_bb7d_9647c88ca50a.slice/crio-c40018903329a6a3315e601743c31013057bf90472c96ac1029cf747c8372a0a WatchSource:0}: Error finding container c40018903329a6a3315e601743c31013057bf90472c96ac1029cf747c8372a0a: Status 404 returned error can't find the container with id c40018903329a6a3315e601743c31013057bf90472c96ac1029cf747c8372a0a Jul 20 19:31:47.171059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:47.171020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:47.171059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:47.171060 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:47.171282 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:47.171135 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:47.171282 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:47.171167 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:47.171282 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:47.171186 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:51.171169278 +0000 UTC m=+252.796248249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : configmap "kube-rbac-proxy" not found Jul 20 19:31:47.171282 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:47.171220 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:51.171204298 +0000 UTC m=+252.796283271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:47.197938 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:47.197913 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qmn6p_ce9b176e-f8c6-45de-bd20-fc9f8b9cf013/serve-healthcheck-canary/0.log" Jul 20 19:31:47.675033 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:47.674998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:47.675513 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:47.675179 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 19:31:47.675513 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:47.675263 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:55.675242436 +0000 UTC m=+257.300321413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : secret "router-metrics-certs-default" not found Jul 20 19:31:47.762379 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:47.762342 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-m9fjk" event={"ID":"06a6cfe6-0400-460c-bb7d-9647c88ca50a","Type":"ContainerStarted","Data":"c40018903329a6a3315e601743c31013057bf90472c96ac1029cf747c8372a0a"} Jul 20 19:31:49.772233 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:49.772192 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-m9fjk" event={"ID":"06a6cfe6-0400-460c-bb7d-9647c88ca50a","Type":"ContainerStarted","Data":"d4b3900266fa13f39cc65be7246807ee078cb0d8bd7a2ec54caebdcc0155c4f5"} Jul 20 19:31:49.788242 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:49.788190 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-m9fjk" podStartSLOduration=1.770352632 podStartE2EDuration="3.788173195s" totalImagesPullingTime="2.017820563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:49.788103028 +0000 UTC m=+251.413182024" watchObservedRunningTime="2026-07-20 19:31:49.788173195 +0000 UTC m=+251.413252189" Jul 20 19:31:50.603098 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:50.603053 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:31:50.603283 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:50.603247 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 19:31:50.603371 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:50.603359 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs podName:a7cfe41a-907b-49d4-89c1-be02103ea048 nodeName:}" failed. No retries permitted until 2026-07-20 19:33:52.603337985 +0000 UTC m=+374.228416961 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs") pod "network-metrics-daemon-b76ts" (UID: "a7cfe41a-907b-49d4-89c1-be02103ea048") : secret "metrics-daemon-secret" not found Jul 20 19:31:51.209018 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:51.208979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:51.209018 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:51.209023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:51.209482 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:51.209141 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:51.209482 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:51.209157 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:51.209482 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:51.209211 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:59.209195595 +0000 UTC m=+260.834274567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:51.209482 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:51.209226 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:31:59.20922032 +0000 UTC m=+260.834299291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : configmap "kube-rbac-proxy" not found Jul 20 19:31:54.837030 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:54.836979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:54.839411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:54.839388 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"image-registry-79d667788f-h69c8\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:54.937522 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:54.937485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:54.937703 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:54.937684 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle podName:100f1256-c5f4-4a3b-bc90-daba6644692c nodeName:}" failed. No retries permitted until 2026-07-20 19:32:10.937663893 +0000 UTC m=+272.562742870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle") pod "router-default-59844479fb-ld75n" (UID: "100f1256-c5f4-4a3b-bc90-daba6644692c") : configmap references non-existent config key: service-ca.crt Jul 20 19:31:54.947185 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:54.947161 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:55.072586 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:55.072553 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79d667788f-h69c8"] Jul 20 19:31:55.075380 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:31:55.075355 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a905de_4b29_429d_b472_41d5c51557b2.slice/crio-1d486af492d007b595043fd30564595d9c3f237b69209b8a0a2af7bfdc64f745 WatchSource:0}: Error finding container 1d486af492d007b595043fd30564595d9c3f237b69209b8a0a2af7bfdc64f745: Status 404 returned error can't find the container with id 1d486af492d007b595043fd30564595d9c3f237b69209b8a0a2af7bfdc64f745 Jul 20 19:31:55.742723 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:55.742685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:55.745076 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:55.745052 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/100f1256-c5f4-4a3b-bc90-daba6644692c-metrics-certs\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:31:55.792177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:55.792145 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d667788f-h69c8" event={"ID":"99a905de-4b29-429d-b472-41d5c51557b2","Type":"ContainerStarted","Data":"ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe"} Jul 20 19:31:55.792346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:55.792193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d667788f-h69c8" event={"ID":"99a905de-4b29-429d-b472-41d5c51557b2","Type":"ContainerStarted","Data":"1d486af492d007b595043fd30564595d9c3f237b69209b8a0a2af7bfdc64f745"} Jul 20 19:31:55.792346 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:55.792220 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:31:55.810265 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:55.810205 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79d667788f-h69c8" podStartSLOduration=16.810189672 podStartE2EDuration="16.810189672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:55.810156848 +0000 UTC m=+257.435235836" watchObservedRunningTime="2026-07-20 19:31:55.810189672 +0000 UTC m=+257.435268665" Jul 20 19:31:59.275410 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:59.275359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:59.275410 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:59.275413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:31:59.275874 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:59.275506 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:59.275874 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:31:59.275590 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls podName:bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd nodeName:}" failed. No retries permitted until 2026-07-20 19:32:15.275571022 +0000 UTC m=+276.900650010 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rdtp" (UID: "bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:59.275978 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:31:59.275958 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:32:07.875718 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:07.875637 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79d667788f-h69c8"] Jul 20 19:32:07.920024 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:07.919985 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dd6ffbd9-rd87d"] Jul 20 19:32:07.924601 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:07.924577 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:07.936704 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:07.936680 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd6ffbd9-rd87d"] Jul 20 19:32:08.047218 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2791edcd-bf0d-4010-8bb6-ee25f69aab36-ca-trust-extracted\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.047368 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-bound-sa-token\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.047368 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2791edcd-bf0d-4010-8bb6-ee25f69aab36-trusted-ca\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.047368 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2791edcd-bf0d-4010-8bb6-ee25f69aab36-installation-pull-secrets\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.047368 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2791edcd-bf0d-4010-8bb6-ee25f69aab36-image-registry-private-configuration\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.047620 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjxbc\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-kube-api-access-cjxbc\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.047620 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2791edcd-bf0d-4010-8bb6-ee25f69aab36-registry-certificates\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.047620 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.047553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-registry-tls\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149000 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.148917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjxbc\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-kube-api-access-cjxbc\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149000 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.148954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2791edcd-bf0d-4010-8bb6-ee25f69aab36-registry-certificates\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149000 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.148973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-registry-tls\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149209 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.149155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2791edcd-bf0d-4010-8bb6-ee25f69aab36-ca-trust-extracted\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149209 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.149196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-bound-sa-token\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149292 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.149264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2791edcd-bf0d-4010-8bb6-ee25f69aab36-trusted-ca\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149336 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.149288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2791edcd-bf0d-4010-8bb6-ee25f69aab36-installation-pull-secrets\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149391 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.149341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2791edcd-bf0d-4010-8bb6-ee25f69aab36-image-registry-private-configuration\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149748 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.149723 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2791edcd-bf0d-4010-8bb6-ee25f69aab36-ca-trust-extracted\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.149945 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.149896 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2791edcd-bf0d-4010-8bb6-ee25f69aab36-registry-certificates\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.150312 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.150290 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2791edcd-bf0d-4010-8bb6-ee25f69aab36-trusted-ca\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.151660 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.151639 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2791edcd-bf0d-4010-8bb6-ee25f69aab36-image-registry-private-configuration\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.151749 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.151691 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-registry-tls\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.151833 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.151811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2791edcd-bf0d-4010-8bb6-ee25f69aab36-installation-pull-secrets\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.156311 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.156284 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-bound-sa-token\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.156760 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.156739 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjxbc\" (UniqueName: \"kubernetes.io/projected/2791edcd-bf0d-4010-8bb6-ee25f69aab36-kube-api-access-cjxbc\") pod \"image-registry-dd6ffbd9-rd87d\" (UID: \"2791edcd-bf0d-4010-8bb6-ee25f69aab36\") " pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.233509 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.233485 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.359227 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.359103 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd6ffbd9-rd87d"] Jul 20 19:32:08.361649 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:32:08.361621 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2791edcd_bf0d_4010_8bb6_ee25f69aab36.slice/crio-4d8d225b3919e169048ee346c1596fd3536415f37818429e0c8a3b87603ceb44 WatchSource:0}: Error finding container 4d8d225b3919e169048ee346c1596fd3536415f37818429e0c8a3b87603ceb44: Status 404 returned error can't find the container with id 4d8d225b3919e169048ee346c1596fd3536415f37818429e0c8a3b87603ceb44 Jul 20 19:32:08.834329 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.834293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" event={"ID":"2791edcd-bf0d-4010-8bb6-ee25f69aab36","Type":"ContainerStarted","Data":"86f0096f120a832fe7e17faccc532c208a60061c594888d83cd10cf614bdf1cf"} Jul 20 19:32:08.834513 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.834350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" event={"ID":"2791edcd-bf0d-4010-8bb6-ee25f69aab36","Type":"ContainerStarted","Data":"4d8d225b3919e169048ee346c1596fd3536415f37818429e0c8a3b87603ceb44"} Jul 20 19:32:08.834513 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.834415 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:08.854040 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:08.853992 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" podStartSLOduration=1.853978058 podStartE2EDuration="1.853978058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:08.85234512 +0000 UTC m=+270.477424115" watchObservedRunningTime="2026-07-20 19:32:08.853978058 +0000 UTC m=+270.479057051" Jul 20 19:32:10.971695 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:10.971646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:32:10.972267 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:10.972246 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/100f1256-c5f4-4a3b-bc90-daba6644692c-service-ca-bundle\") pod \"router-default-59844479fb-ld75n\" (UID: \"100f1256-c5f4-4a3b-bc90-daba6644692c\") " pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:32:11.255350 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:11.255274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:32:11.383632 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:11.383608 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59844479fb-ld75n"] Jul 20 19:32:11.385910 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:32:11.385884 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100f1256_c5f4_4a3b_bc90_daba6644692c.slice/crio-1e85dd2f66dfd923df28ec740d66bf27571b2a93dae2062928b579ab6c12a6b8 WatchSource:0}: Error finding container 1e85dd2f66dfd923df28ec740d66bf27571b2a93dae2062928b579ab6c12a6b8: Status 404 returned error can't find the container with id 1e85dd2f66dfd923df28ec740d66bf27571b2a93dae2062928b579ab6c12a6b8 Jul 20 19:32:11.844842 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:11.844806 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59844479fb-ld75n" event={"ID":"100f1256-c5f4-4a3b-bc90-daba6644692c","Type":"ContainerStarted","Data":"b10fd8ebaf0ced12bd4e4d9877c2e649c37ad635e5fd8032fb2b3c9fb43fdda7"} Jul 20 19:32:11.845011 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:11.844854 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59844479fb-ld75n" event={"ID":"100f1256-c5f4-4a3b-bc90-daba6644692c","Type":"ContainerStarted","Data":"1e85dd2f66dfd923df28ec740d66bf27571b2a93dae2062928b579ab6c12a6b8"} Jul 20 19:32:11.863668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:11.863622 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-59844479fb-ld75n" podStartSLOduration=32.863607596 podStartE2EDuration="32.863607596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:11.862281239 +0000 UTC m=+273.487360224" watchObservedRunningTime="2026-07-20 19:32:11.863607596 +0000 UTC m=+273.488686589" Jul 20 19:32:12.255644 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:12.255562 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:32:12.258480 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:12.258454 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:32:12.847755 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:12.847724 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:32:12.848950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:12.848924 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59844479fb-ld75n" Jul 20 19:32:15.303273 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:15.303238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:32:15.305549 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:15.305525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rdtp\" (UID: \"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd\") " pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:32:15.534438 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:15.534392 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4rdtp" Jul 20 19:32:15.660056 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:15.659852 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4rdtp"] Jul 20 19:32:15.662697 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:32:15.662659 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe760d1_64f1_4424_9a7d_cf3c6aa5a8dd.slice/crio-becf4feb1b34b0970053894c6fc5be3fbbe5c6adc81db6101fa504f77bd0dac1 WatchSource:0}: Error finding container becf4feb1b34b0970053894c6fc5be3fbbe5c6adc81db6101fa504f77bd0dac1: Status 404 returned error can't find the container with id becf4feb1b34b0970053894c6fc5be3fbbe5c6adc81db6101fa504f77bd0dac1 Jul 20 19:32:15.856783 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:15.856699 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rdtp" event={"ID":"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd","Type":"ContainerStarted","Data":"becf4feb1b34b0970053894c6fc5be3fbbe5c6adc81db6101fa504f77bd0dac1"} Jul 20 19:32:17.883497 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:17.883463 2580 patch_prober.go:28] interesting pod/image-registry-79d667788f-h69c8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 19:32:17.883911 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:17.883532 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-79d667788f-h69c8" podUID="99a905de-4b29-429d-b472-41d5c51557b2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 19:32:20.880175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:20.880078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rdtp" event={"ID":"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd","Type":"ContainerStarted","Data":"89d7862e5a0b27730777c1591f01aa15c13a5885f8b1e2e7ff61de46b25de345"} Jul 20 19:32:21.537592 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:32:21.537522 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tvwsq" podUID="c967fb7b-6a3f-428a-8600-a7c39c1973af" Jul 20 19:32:21.883975 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:21.883898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tvwsq" Jul 20 19:32:23.218443 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.218393 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q4fz9"] Jul 20 19:32:23.238456 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.238380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.240839 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.240631 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 19:32:23.241252 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.241112 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 19:32:23.241252 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.241127 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 19:32:23.241407 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.241352 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r5lrj\"" Jul 20 19:32:23.241728 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.241548 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 19:32:23.241728 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.241576 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 19:32:23.242272 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.242112 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 19:32:23.280496 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-tls\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280645 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280645 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280576 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m4jz\" (UniqueName: \"kubernetes.io/projected/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-kube-api-access-7m4jz\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280645 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-wtmp\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280820 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-accelerators-collector-config\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280820 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-textfile\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280820 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280699 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-root\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280820 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-sys\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.280820 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.280769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-metrics-client-ca\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381379 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-tls\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381574 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381574 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m4jz\" (UniqueName: \"kubernetes.io/projected/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-kube-api-access-7m4jz\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381574 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:32:23.381515 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 19:32:23.381574 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-wtmp\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381574 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381562 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-accelerators-collector-config\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381836 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:32:23.381581 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-tls podName:d9b8dcfc-d329-49e4-8c22-554e1f8a6acd nodeName:}" failed. No retries permitted until 2026-07-20 19:32:23.881561159 +0000 UTC m=+285.506640146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-tls") pod "node-exporter-q4fz9" (UID: "d9b8dcfc-d329-49e4-8c22-554e1f8a6acd") : secret "node-exporter-tls" not found Jul 20 19:32:23.381836 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-textfile\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381836 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-root\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381836 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-sys\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.381836 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.381725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-metrics-client-ca\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.382386 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.382127 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-accelerators-collector-config\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.382386 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.382273 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-wtmp\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.382386 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.382290 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-root\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.382386 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.382322 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-metrics-client-ca\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.382386 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.382341 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-sys\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.382655 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.382522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-textfile\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.384209 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.384185 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.404953 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.404926 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m4jz\" (UniqueName: \"kubernetes.io/projected/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-kube-api-access-7m4jz\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.887069 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.887036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-tls\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:23.889572 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:23.889543 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9b8dcfc-d329-49e4-8c22-554e1f8a6acd-node-exporter-tls\") pod \"node-exporter-q4fz9\" (UID: \"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd\") " pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:24.150894 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:24.150809 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q4fz9" Jul 20 19:32:24.292566 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:32:24.292530 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b8dcfc_d329_49e4_8c22_554e1f8a6acd.slice/crio-5a21ce009cda7a018cf57f8d69b2741ee100007fd04ae6361724e9cc40510528 WatchSource:0}: Error finding container 5a21ce009cda7a018cf57f8d69b2741ee100007fd04ae6361724e9cc40510528: Status 404 returned error can't find the container with id 5a21ce009cda7a018cf57f8d69b2741ee100007fd04ae6361724e9cc40510528 Jul 20 19:32:24.593750 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:24.593722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:32:24.595983 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:24.595960 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c967fb7b-6a3f-428a-8600-a7c39c1973af-metrics-tls\") pod \"dns-default-tvwsq\" (UID: \"c967fb7b-6a3f-428a-8600-a7c39c1973af\") " pod="openshift-dns/dns-default-tvwsq" Jul 20 19:32:24.886992 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:24.886913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27sg8\"" Jul 20 19:32:24.895320 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:24.895241 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tvwsq" Jul 20 19:32:24.898761 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:24.898694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q4fz9" event={"ID":"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd","Type":"ContainerStarted","Data":"5a21ce009cda7a018cf57f8d69b2741ee100007fd04ae6361724e9cc40510528"} Jul 20 19:32:24.900924 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:24.900860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rdtp" event={"ID":"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd","Type":"ContainerStarted","Data":"b7a523e4af834ff033081f55d271c29315fd56a9aba25f3feb73bf9da0df7f29"} Jul 20 19:32:25.042711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:25.042617 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tvwsq"] Jul 20 19:32:25.046610 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:32:25.046560 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc967fb7b_6a3f_428a_8600_a7c39c1973af.slice/crio-175da34ba69ed5c0aad96edbcb2987e646f98cbad53ec4af6f2c0bf439617c6a WatchSource:0}: Error finding container 175da34ba69ed5c0aad96edbcb2987e646f98cbad53ec4af6f2c0bf439617c6a: Status 404 returned error can't find the container with id 175da34ba69ed5c0aad96edbcb2987e646f98cbad53ec4af6f2c0bf439617c6a Jul 20 19:32:25.907373 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:25.907335 2580 generic.go:356] "Generic (PLEG): container finished" podID="d9b8dcfc-d329-49e4-8c22-554e1f8a6acd" containerID="0c30d6f306ca17bb6632f0220aa4c124aa0a1006ec3a9baf1c76a9ef13fb892d" exitCode=0 Jul 20 19:32:25.907872 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:25.907439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q4fz9" event={"ID":"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd","Type":"ContainerDied","Data":"0c30d6f306ca17bb6632f0220aa4c124aa0a1006ec3a9baf1c76a9ef13fb892d"} Jul 20 19:32:25.908748 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:25.908725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tvwsq" event={"ID":"c967fb7b-6a3f-428a-8600-a7c39c1973af","Type":"ContainerStarted","Data":"175da34ba69ed5c0aad96edbcb2987e646f98cbad53ec4af6f2c0bf439617c6a"} Jul 20 19:32:26.914640 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:26.914600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q4fz9" event={"ID":"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd","Type":"ContainerStarted","Data":"1760ed869ef1cfafa0257ed8a9a06dd3ea00c3ad4a203a2dca60ef5b51fb84c0"} Jul 20 19:32:26.915128 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:26.914688 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q4fz9" event={"ID":"d9b8dcfc-d329-49e4-8c22-554e1f8a6acd","Type":"ContainerStarted","Data":"3bb3d38b1f9d0951969cb97283b6ed88c28f2b46b471596b851f398d6e75cf80"} Jul 20 19:32:26.931566 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:26.931521 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q4fz9" podStartSLOduration=2.765814223 podStartE2EDuration="3.931501113s" totalImagesPullingTime="1.088251049s" totalInitContainerRuntime="77.435841ms" isStatefulPod=true podCreationTimestamp="2026-07-20 19:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:26.93060794 +0000 UTC m=+288.555686933" watchObservedRunningTime="2026-07-20 19:32:26.931501113 +0000 UTC m=+288.556580109" Jul 20 19:32:27.881909 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:27.881873 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:32:27.963116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:27.963077 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6"] Jul 20 19:32:27.990178 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:27.990145 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6"] Jul 20 19:32:27.990335 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:27.990278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:27.992558 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:27.992533 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kmnz9\"" Jul 20 19:32:27.992711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:27.992562 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 20 19:32:28.126484 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.126445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7e7a011-b8d6-463d-9da0-f4353591ce25-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-n72w6\" (UID: \"c7e7a011-b8d6-463d-9da0-f4353591ce25\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:28.126647 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.126592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5k5s\" (UniqueName: \"kubernetes.io/projected/c7e7a011-b8d6-463d-9da0-f4353591ce25-kube-api-access-h5k5s\") pod \"monitoring-plugin-7b7b7bc8bf-n72w6\" (UID: \"c7e7a011-b8d6-463d-9da0-f4353591ce25\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:28.227605 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.227519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7e7a011-b8d6-463d-9da0-f4353591ce25-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-n72w6\" (UID: \"c7e7a011-b8d6-463d-9da0-f4353591ce25\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:28.227771 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.227641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5k5s\" (UniqueName: \"kubernetes.io/projected/c7e7a011-b8d6-463d-9da0-f4353591ce25-kube-api-access-h5k5s\") pod \"monitoring-plugin-7b7b7bc8bf-n72w6\" (UID: \"c7e7a011-b8d6-463d-9da0-f4353591ce25\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:28.230272 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.230243 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7e7a011-b8d6-463d-9da0-f4353591ce25-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-n72w6\" (UID: \"c7e7a011-b8d6-463d-9da0-f4353591ce25\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:28.235647 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.235594 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5k5s\" (UniqueName: \"kubernetes.io/projected/c7e7a011-b8d6-463d-9da0-f4353591ce25-kube-api-access-h5k5s\") pod \"monitoring-plugin-7b7b7bc8bf-n72w6\" (UID: \"c7e7a011-b8d6-463d-9da0-f4353591ce25\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:28.239801 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.239771 2580 patch_prober.go:28] interesting pod/image-registry-dd6ffbd9-rd87d container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 19:32:28.239929 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.239823 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" podUID="2791edcd-bf0d-4010-8bb6-ee25f69aab36" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 19:32:28.300814 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.300777 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:28.711149 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.711087 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6"] Jul 20 19:32:28.714217 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:32:28.714190 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7e7a011_b8d6_463d_9da0_f4353591ce25.slice/crio-54e36b3dcc259757ae983b2c206030b5f3c449439bd9a09c9bbad89b1d4b5dd9 WatchSource:0}: Error finding container 54e36b3dcc259757ae983b2c206030b5f3c449439bd9a09c9bbad89b1d4b5dd9: Status 404 returned error can't find the container with id 54e36b3dcc259757ae983b2c206030b5f3c449439bd9a09c9bbad89b1d4b5dd9 Jul 20 19:32:28.925025 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.924923 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tvwsq" event={"ID":"c967fb7b-6a3f-428a-8600-a7c39c1973af","Type":"ContainerStarted","Data":"741d5acc70ba9021730eb1cf7c214373c1e9edba1e31500201680b1e243681c6"} Jul 20 19:32:28.925025 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.924990 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tvwsq" event={"ID":"c967fb7b-6a3f-428a-8600-a7c39c1973af","Type":"ContainerStarted","Data":"e8d1c775c4dd8472dec07ccc94eb330c31799f2b0cd28106848f56c4e8e553b6"} Jul 20 19:32:28.925246 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.925063 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tvwsq" Jul 20 19:32:28.926073 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.926045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" event={"ID":"c7e7a011-b8d6-463d-9da0-f4353591ce25","Type":"ContainerStarted","Data":"54e36b3dcc259757ae983b2c206030b5f3c449439bd9a09c9bbad89b1d4b5dd9"} Jul 20 19:32:28.927753 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.927730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rdtp" event={"ID":"bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd","Type":"ContainerStarted","Data":"74c64c38a02269b4ef6a685c061b2d2bb3c52ce29accc79eb765fa1bc0208439"} Jul 20 19:32:28.942516 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.942475 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tvwsq" podStartSLOduration=251.404630719 podStartE2EDuration="4m14.942461792s" totalImagesPullingTime="3.537831073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:28.941063724 +0000 UTC m=+290.566142721" watchObservedRunningTime="2026-07-20 19:32:28.942461792 +0000 UTC m=+290.567540785" Jul 20 19:32:28.956654 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:28.956612 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4rdtp" podStartSLOduration=33.148399022 podStartE2EDuration="45.956598979s" totalImagesPullingTime="12.808199957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:31:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:28.956021793 +0000 UTC m=+290.581100789" watchObservedRunningTime="2026-07-20 19:32:28.956598979 +0000 UTC m=+290.581677973" Jul 20 19:32:29.842581 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:29.842552 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dd6ffbd9-rd87d" Jul 20 19:32:30.937275 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:30.937240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" event={"ID":"c7e7a011-b8d6-463d-9da0-f4353591ce25","Type":"ContainerStarted","Data":"21f56e419b949e8e3cb0ee95450cbd5336742bf5a5010996c02d280327e46390"} Jul 20 19:32:30.937770 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:30.937417 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:30.942002 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:30.941975 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" Jul 20 19:32:30.953439 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:30.953380 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-n72w6" podStartSLOduration=2.487191541 podStartE2EDuration="3.953362396s" totalImagesPullingTime="1.466170855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:30.952412861 +0000 UTC m=+292.577491855" watchObservedRunningTime="2026-07-20 19:32:30.953362396 +0000 UTC m=+292.578441391" Jul 20 19:32:32.894907 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:32.894859 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79d667788f-h69c8" podUID="99a905de-4b29-429d-b472-41d5c51557b2" containerName="registry" containerID="cri-o://ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe" gracePeriod=30 Jul 20 19:32:33.132675 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.132650 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:32:33.272355 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272270 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272355 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272311 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-registry-certificates\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272355 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272328 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-trusted-ca\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272355 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272350 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99a905de-4b29-429d-b472-41d5c51557b2-ca-trust-extracted\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272376 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-image-registry-private-configuration\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272399 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-installation-pull-secrets\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272468 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz7ls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-kube-api-access-cz7ls\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272527 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-bound-sa-token\") pod \"99a905de-4b29-429d-b472-41d5c51557b2\" (UID: \"99a905de-4b29-429d-b472-41d5c51557b2\") " Jul 20 19:32:33.272960 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272913 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:32:33.273085 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.272914 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:32:33.275203 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.275180 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:32:33.275274 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.275205 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:32:33.275324 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.275268 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:32:33.275324 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.275282 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:32:33.275391 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.275320 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-kube-api-access-cz7ls" (OuterVolumeSpecName: "kube-api-access-cz7ls") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "kube-api-access-cz7ls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:32:33.281798 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.281777 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a905de-4b29-429d-b472-41d5c51557b2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "99a905de-4b29-429d-b472-41d5c51557b2" (UID: "99a905de-4b29-429d-b472-41d5c51557b2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:32:33.373417 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373383 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-registry-certificates\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.373417 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373413 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/99a905de-4b29-429d-b472-41d5c51557b2-trusted-ca\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.373594 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373440 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/99a905de-4b29-429d-b472-41d5c51557b2-ca-trust-extracted\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.373594 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373450 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-image-registry-private-configuration\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.373594 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373460 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/99a905de-4b29-429d-b472-41d5c51557b2-installation-pull-secrets\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.373594 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373468 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz7ls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-kube-api-access-cz7ls\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.373594 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373476 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-bound-sa-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.373594 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.373484 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/99a905de-4b29-429d-b472-41d5c51557b2-registry-tls\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:32:33.949568 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.949530 2580 generic.go:356] "Generic (PLEG): container finished" podID="99a905de-4b29-429d-b472-41d5c51557b2" containerID="ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe" exitCode=0 Jul 20 19:32:33.949990 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.949594 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79d667788f-h69c8" Jul 20 19:32:33.949990 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.949611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d667788f-h69c8" event={"ID":"99a905de-4b29-429d-b472-41d5c51557b2","Type":"ContainerDied","Data":"ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe"} Jul 20 19:32:33.949990 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.949689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79d667788f-h69c8" event={"ID":"99a905de-4b29-429d-b472-41d5c51557b2","Type":"ContainerDied","Data":"1d486af492d007b595043fd30564595d9c3f237b69209b8a0a2af7bfdc64f745"} Jul 20 19:32:33.949990 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.949719 2580 scope.go:117] "RemoveContainer" containerID="ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe" Jul 20 19:32:33.958633 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.958613 2580 scope.go:117] "RemoveContainer" containerID="ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe" Jul 20 19:32:33.958945 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:32:33.958924 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe\": container with ID starting with ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe not found: ID does not exist" containerID="ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe" Jul 20 19:32:33.958988 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.958956 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe"} err="failed to get container status \"ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe\": rpc error: code = NotFound desc = could not find container \"ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe\": container with ID starting with ab0ebdb5d827736c8f962fac409f15bfa9870eebd084e777a6b789981d680bbe not found: ID does not exist" Jul 20 19:32:33.970179 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.970150 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79d667788f-h69c8"] Jul 20 19:32:33.976951 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:33.976924 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79d667788f-h69c8"] Jul 20 19:32:35.059492 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:35.059457 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99a905de-4b29-429d-b472-41d5c51557b2" path="/var/lib/kubelet/pods/99a905de-4b29-429d-b472-41d5c51557b2/volumes" Jul 20 19:32:36.056538 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:32:36.056482 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b76ts" podUID="a7cfe41a-907b-49d4-89c1-be02103ea048" Jul 20 19:32:38.751039 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:38.751019 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 19:32:38.933961 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:38.933927 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tvwsq" Jul 20 19:32:39.998309 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:39.998276 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57cd5bb844-2lscx"] Jul 20 19:32:39.999842 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:39.998525 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99a905de-4b29-429d-b472-41d5c51557b2" containerName="registry" Jul 20 19:32:39.999842 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:39.998536 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a905de-4b29-429d-b472-41d5c51557b2" containerName="registry" Jul 20 19:32:39.999842 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:39.998584 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="99a905de-4b29-429d-b472-41d5c51557b2" containerName="registry" Jul 20 19:32:40.000158 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.000142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.002175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.002155 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 19:32:40.003033 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.003012 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 19:32:40.003033 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.003030 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 19:32:40.003186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.003097 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dnpwh\"" Jul 20 19:32:40.003186 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.003102 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 19:32:40.003334 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.003319 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 19:32:40.003395 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.003384 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 19:32:40.003698 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.003675 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 19:32:40.010207 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.010189 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 19:32:40.013907 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.013887 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57cd5bb844-2lscx"] Jul 20 19:32:40.129253 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.129214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-serving-cert\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.129468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.129271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xztt9\" (UniqueName: \"kubernetes.io/projected/5346d0c9-8691-4c1a-a06c-8e3667453eb5-kube-api-access-xztt9\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.129468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.129330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-trusted-ca-bundle\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.129468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.129401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-config\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.129468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.129461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-service-ca\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.129644 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.129488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-oauth-config\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.129644 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.129513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-oauth-serving-cert\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.230230 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.230194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-service-ca\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.230230 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.230233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-oauth-config\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.230420 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.230252 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-oauth-serving-cert\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.230420 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.230272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-serving-cert\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.230420 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.230396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xztt9\" (UniqueName: \"kubernetes.io/projected/5346d0c9-8691-4c1a-a06c-8e3667453eb5-kube-api-access-xztt9\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.230584 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.230500 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-trusted-ca-bundle\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.230584 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.230556 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-config\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.231056 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.231028 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-service-ca\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.231056 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.231053 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-oauth-serving-cert\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.231226 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.231138 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-config\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.231452 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.231412 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-trusted-ca-bundle\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.232691 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.232662 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-serving-cert\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.232789 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.232695 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-oauth-config\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.238484 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.238463 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xztt9\" (UniqueName: \"kubernetes.io/projected/5346d0c9-8691-4c1a-a06c-8e3667453eb5-kube-api-access-xztt9\") pod \"console-57cd5bb844-2lscx\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.310073 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.310045 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:40.431555 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.431525 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57cd5bb844-2lscx"] Jul 20 19:32:40.435179 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:32:40.435148 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5346d0c9_8691_4c1a_a06c_8e3667453eb5.slice/crio-6d5708c5818486fffeef53aa499c6ad0903b283625b1eeeb1d110a926e1ea734 WatchSource:0}: Error finding container 6d5708c5818486fffeef53aa499c6ad0903b283625b1eeeb1d110a926e1ea734: Status 404 returned error can't find the container with id 6d5708c5818486fffeef53aa499c6ad0903b283625b1eeeb1d110a926e1ea734 Jul 20 19:32:40.974769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:40.974724 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57cd5bb844-2lscx" event={"ID":"5346d0c9-8691-4c1a-a06c-8e3667453eb5","Type":"ContainerStarted","Data":"6d5708c5818486fffeef53aa499c6ad0903b283625b1eeeb1d110a926e1ea734"} Jul 20 19:32:43.988165 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:43.988125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57cd5bb844-2lscx" event={"ID":"5346d0c9-8691-4c1a-a06c-8e3667453eb5","Type":"ContainerStarted","Data":"5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8"} Jul 20 19:32:44.009095 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:44.008981 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57cd5bb844-2lscx" podStartSLOduration=2.159595774 podStartE2EDuration="5.008960752s" totalImagesPullingTime="2.849364978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:44.008658848 +0000 UTC m=+305.633737842" watchObservedRunningTime="2026-07-20 19:32:44.008960752 +0000 UTC m=+305.634039747" Jul 20 19:32:47.059380 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:47.059348 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:32:49.005229 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:49.005192 2580 generic.go:356] "Generic (PLEG): container finished" podID="ec1ccec2-98b0-435d-9e8e-c2de5706aadd" containerID="b5dc65d66f1603bc762d8f80925144c39c0186e1479fce35ea62fa1472b2a048" exitCode=0 Jul 20 19:32:49.005653 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:49.005235 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qqmfb" event={"ID":"ec1ccec2-98b0-435d-9e8e-c2de5706aadd","Type":"ContainerDied","Data":"b5dc65d66f1603bc762d8f80925144c39c0186e1479fce35ea62fa1472b2a048"} Jul 20 19:32:49.005653 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:49.005561 2580 scope.go:117] "RemoveContainer" containerID="b5dc65d66f1603bc762d8f80925144c39c0186e1479fce35ea62fa1472b2a048" Jul 20 19:32:50.011315 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:50.011282 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-qqmfb" event={"ID":"ec1ccec2-98b0-435d-9e8e-c2de5706aadd","Type":"ContainerStarted","Data":"94d5f225d4846d337df504306eb829a4660042ea4264ff31e9dcb423735fe33b"} Jul 20 19:32:50.310880 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:50.310846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:50.311054 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:50.310936 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:50.315722 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:50.315699 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:51.018470 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:51.018416 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:32:53.022137 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:53.022054 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef772c49-d191-4c78-b4fe-13512691a076" containerID="81e8c5104e82c1f76b167f95f93a2abce9a389c46f1aa715adabd9c284649edf" exitCode=0 Jul 20 19:32:53.022579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:53.022130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" event={"ID":"ef772c49-d191-4c78-b4fe-13512691a076","Type":"ContainerDied","Data":"81e8c5104e82c1f76b167f95f93a2abce9a389c46f1aa715adabd9c284649edf"} Jul 20 19:32:53.022579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:53.022488 2580 scope.go:117] "RemoveContainer" containerID="81e8c5104e82c1f76b167f95f93a2abce9a389c46f1aa715adabd9c284649edf" Jul 20 19:32:54.026941 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:32:54.026903 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-75wvc" event={"ID":"ef772c49-d191-4c78-b4fe-13512691a076","Type":"ContainerStarted","Data":"23bacc3650b13f7901e1eaa1b97f4282ed70fe1b99b5acd34653cb35db21bd28"} Jul 20 19:33:43.236505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.236407 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c94dfd697-nvzf9"] Jul 20 19:33:43.239770 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.239746 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.252353 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.252326 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c94dfd697-nvzf9"] Jul 20 19:33:43.265504 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.265473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-console-config\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.265504 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.265516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-service-ca\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.265685 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.265573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhp9j\" (UniqueName: \"kubernetes.io/projected/420121bb-5288-4e84-9d0e-baabe508d3d7-kube-api-access-rhp9j\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.265685 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.265617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-oauth-serving-cert\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.265685 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.265636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-trusted-ca-bundle\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.265833 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.265723 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-serving-cert\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.265833 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.265765 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-oauth-config\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.366858 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.366825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-oauth-serving-cert\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.366858 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.366861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-trusted-ca-bundle\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367071 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.366916 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-serving-cert\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367071 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.366947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-oauth-config\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367071 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.367003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-console-config\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367071 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.367029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-service-ca\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367071 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.367055 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhp9j\" (UniqueName: \"kubernetes.io/projected/420121bb-5288-4e84-9d0e-baabe508d3d7-kube-api-access-rhp9j\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367719 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.367692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-oauth-serving-cert\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367866 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.367760 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-console-config\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367940 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.367910 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-service-ca\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.367999 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.367918 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-trusted-ca-bundle\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.369500 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.369478 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-serving-cert\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.369610 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.369506 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-oauth-config\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.374120 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.374099 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhp9j\" (UniqueName: \"kubernetes.io/projected/420121bb-5288-4e84-9d0e-baabe508d3d7-kube-api-access-rhp9j\") pod \"console-6c94dfd697-nvzf9\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.552444 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.551620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:43.688866 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:43.688831 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c94dfd697-nvzf9"] Jul 20 19:33:43.691576 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:33:43.691546 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420121bb_5288_4e84_9d0e_baabe508d3d7.slice/crio-0012f34962250d1727342fe6645c56599066f0b7d00f49ec9771fc799241d0b9 WatchSource:0}: Error finding container 0012f34962250d1727342fe6645c56599066f0b7d00f49ec9771fc799241d0b9: Status 404 returned error can't find the container with id 0012f34962250d1727342fe6645c56599066f0b7d00f49ec9771fc799241d0b9 Jul 20 19:33:44.196646 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:44.196610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c94dfd697-nvzf9" event={"ID":"420121bb-5288-4e84-9d0e-baabe508d3d7","Type":"ContainerStarted","Data":"4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51"} Jul 20 19:33:44.196819 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:44.196661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c94dfd697-nvzf9" event={"ID":"420121bb-5288-4e84-9d0e-baabe508d3d7","Type":"ContainerStarted","Data":"0012f34962250d1727342fe6645c56599066f0b7d00f49ec9771fc799241d0b9"} Jul 20 19:33:44.212464 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:44.212398 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c94dfd697-nvzf9" podStartSLOduration=1.212381242 podStartE2EDuration="1.212381242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:33:44.211247205 +0000 UTC m=+365.836326199" watchObservedRunningTime="2026-07-20 19:33:44.212381242 +0000 UTC m=+365.837460238" Jul 20 19:33:52.639685 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:52.639641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:33:52.641913 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:52.641888 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cfe41a-907b-49d4-89c1-be02103ea048-metrics-certs\") pod \"network-metrics-daemon-b76ts\" (UID: \"a7cfe41a-907b-49d4-89c1-be02103ea048\") " pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:33:52.764446 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:52.764395 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pw7wf\"" Jul 20 19:33:52.771033 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:52.771016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b76ts" Jul 20 19:33:52.897239 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:52.897214 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b76ts"] Jul 20 19:33:52.899265 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:33:52.899233 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cfe41a_907b_49d4_89c1_be02103ea048.slice/crio-b3d4ad23f506d2414027da22b64951c1542635c7c42c0acdb998532f416ebe25 WatchSource:0}: Error finding container b3d4ad23f506d2414027da22b64951c1542635c7c42c0acdb998532f416ebe25: Status 404 returned error can't find the container with id b3d4ad23f506d2414027da22b64951c1542635c7c42c0acdb998532f416ebe25 Jul 20 19:33:52.900984 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:52.900968 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 19:33:53.228764 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:53.228680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b76ts" event={"ID":"a7cfe41a-907b-49d4-89c1-be02103ea048","Type":"ContainerStarted","Data":"b3d4ad23f506d2414027da22b64951c1542635c7c42c0acdb998532f416ebe25"} Jul 20 19:33:53.552721 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:53.552686 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:53.552951 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:53.552745 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:53.558006 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:53.557983 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:54.233405 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:54.233311 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b76ts" event={"ID":"a7cfe41a-907b-49d4-89c1-be02103ea048","Type":"ContainerStarted","Data":"dc82ad9bec68a2d325031ee44cd27e746478dab8710597bf8ebd93f2a5dc5752"} Jul 20 19:33:54.233405 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:54.233368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b76ts" event={"ID":"a7cfe41a-907b-49d4-89c1-be02103ea048","Type":"ContainerStarted","Data":"7ba5d00ffd50fe0bc3006cf9e78b02ccbdf05f78ae7ad34f2d8274ce17821953"} Jul 20 19:33:54.237291 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:54.237266 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:33:54.251626 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:54.251579 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b76ts" podStartSLOduration=374.429639682 podStartE2EDuration="6m15.251564153s" totalImagesPullingTime="821.924471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:33:54.250356382 +0000 UTC m=+375.875435377" watchObservedRunningTime="2026-07-20 19:33:54.251564153 +0000 UTC m=+375.876643146" Jul 20 19:33:54.298380 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:33:54.298347 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57cd5bb844-2lscx"] Jul 20 19:34:19.326645 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.326596 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57cd5bb844-2lscx" podUID="5346d0c9-8691-4c1a-a06c-8e3667453eb5" containerName="console" containerID="cri-o://5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8" gracePeriod=15 Jul 20 19:34:19.568401 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.568374 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:34:19.659343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659247 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-oauth-config\") pod \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " Jul 20 19:34:19.659343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659291 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-service-ca\") pod \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " Jul 20 19:34:19.659343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659319 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-config\") pod \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " Jul 20 19:34:19.659343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659338 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xztt9\" (UniqueName: \"kubernetes.io/projected/5346d0c9-8691-4c1a-a06c-8e3667453eb5-kube-api-access-xztt9\") pod \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " Jul 20 19:34:19.659717 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659367 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-oauth-serving-cert\") pod \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " Jul 20 19:34:19.659717 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659421 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-trusted-ca-bundle\") pod \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " Jul 20 19:34:19.659717 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659483 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-serving-cert\") pod \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\" (UID: \"5346d0c9-8691-4c1a-a06c-8e3667453eb5\") " Jul 20 19:34:19.659886 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659725 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-service-ca" (OuterVolumeSpecName: "service-ca") pod "5346d0c9-8691-4c1a-a06c-8e3667453eb5" (UID: "5346d0c9-8691-4c1a-a06c-8e3667453eb5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:34:19.659943 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659896 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5346d0c9-8691-4c1a-a06c-8e3667453eb5" (UID: "5346d0c9-8691-4c1a-a06c-8e3667453eb5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:34:19.659999 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659952 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5346d0c9-8691-4c1a-a06c-8e3667453eb5" (UID: "5346d0c9-8691-4c1a-a06c-8e3667453eb5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:34:19.659999 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.659948 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-config" (OuterVolumeSpecName: "console-config") pod "5346d0c9-8691-4c1a-a06c-8e3667453eb5" (UID: "5346d0c9-8691-4c1a-a06c-8e3667453eb5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:34:19.661620 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.661600 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5346d0c9-8691-4c1a-a06c-8e3667453eb5" (UID: "5346d0c9-8691-4c1a-a06c-8e3667453eb5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:34:19.661750 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.661733 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5346d0c9-8691-4c1a-a06c-8e3667453eb5" (UID: "5346d0c9-8691-4c1a-a06c-8e3667453eb5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:34:19.661796 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.661755 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5346d0c9-8691-4c1a-a06c-8e3667453eb5-kube-api-access-xztt9" (OuterVolumeSpecName: "kube-api-access-xztt9") pod "5346d0c9-8691-4c1a-a06c-8e3667453eb5" (UID: "5346d0c9-8691-4c1a-a06c-8e3667453eb5"). InnerVolumeSpecName "kube-api-access-xztt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:34:19.760237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.760191 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-serving-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:34:19.760237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.760226 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-oauth-config\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:34:19.760237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.760235 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-service-ca\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:34:19.760237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.760243 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-console-config\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:34:19.760585 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.760252 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xztt9\" (UniqueName: \"kubernetes.io/projected/5346d0c9-8691-4c1a-a06c-8e3667453eb5-kube-api-access-xztt9\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:34:19.760585 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.760261 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-oauth-serving-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:34:19.760585 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:19.760271 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5346d0c9-8691-4c1a-a06c-8e3667453eb5-trusted-ca-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:34:20.327548 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.327516 2580 generic.go:356] "Generic (PLEG): container finished" podID="5346d0c9-8691-4c1a-a06c-8e3667453eb5" containerID="5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8" exitCode=0 Jul 20 19:34:20.327950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.327557 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57cd5bb844-2lscx" Jul 20 19:34:20.327950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.327596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57cd5bb844-2lscx" event={"ID":"5346d0c9-8691-4c1a-a06c-8e3667453eb5","Type":"ContainerDied","Data":"5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8"} Jul 20 19:34:20.327950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.327633 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57cd5bb844-2lscx" event={"ID":"5346d0c9-8691-4c1a-a06c-8e3667453eb5","Type":"ContainerDied","Data":"6d5708c5818486fffeef53aa499c6ad0903b283625b1eeeb1d110a926e1ea734"} Jul 20 19:34:20.327950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.327653 2580 scope.go:117] "RemoveContainer" containerID="5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8" Jul 20 19:34:20.335952 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.335933 2580 scope.go:117] "RemoveContainer" containerID="5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8" Jul 20 19:34:20.336180 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:34:20.336162 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8\": container with ID starting with 5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8 not found: ID does not exist" containerID="5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8" Jul 20 19:34:20.336224 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.336190 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8"} err="failed to get container status \"5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8\": rpc error: code = NotFound desc = could not find container \"5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8\": container with ID starting with 5a8e138045f91446c8629f7586bc2ee6b498db9f4f8ba81fd461ef2ab69016a8 not found: ID does not exist" Jul 20 19:34:20.346664 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.346639 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57cd5bb844-2lscx"] Jul 20 19:34:20.350895 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:20.350873 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57cd5bb844-2lscx"] Jul 20 19:34:21.060864 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:21.060829 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5346d0c9-8691-4c1a-a06c-8e3667453eb5" path="/var/lib/kubelet/pods/5346d0c9-8691-4c1a-a06c-8e3667453eb5/volumes" Jul 20 19:34:53.817530 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.817495 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r"] Jul 20 19:34:53.817974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.817763 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5346d0c9-8691-4c1a-a06c-8e3667453eb5" containerName="console" Jul 20 19:34:53.817974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.817773 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5346d0c9-8691-4c1a-a06c-8e3667453eb5" containerName="console" Jul 20 19:34:53.817974 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.817825 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5346d0c9-8691-4c1a-a06c-8e3667453eb5" containerName="console" Jul 20 19:34:53.821997 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.821979 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:53.824072 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.824044 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s2256\"" Jul 20 19:34:53.824208 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.824093 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 19:34:53.824509 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.824488 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 19:34:53.830267 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.830240 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r"] Jul 20 19:34:53.926331 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.926287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:53.926331 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.926332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:53.926592 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:53.926410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jnwr\" (UniqueName: \"kubernetes.io/projected/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-kube-api-access-2jnwr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.027603 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.027564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.027603 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.027603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.027856 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.027639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jnwr\" (UniqueName: \"kubernetes.io/projected/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-kube-api-access-2jnwr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.028038 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.028017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.028108 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.028035 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.035956 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.035931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jnwr\" (UniqueName: \"kubernetes.io/projected/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-kube-api-access-2jnwr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.131547 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.131461 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:34:54.259390 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.259363 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r"] Jul 20 19:34:54.261916 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:34:54.261881 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487a54ea_85e1_40b0_b57b_6e6ac7b30d7a.slice/crio-9836e08f7980595f0ddf664ff04f7e3c69d048b10fca1c04c115e4fee7d2e198 WatchSource:0}: Error finding container 9836e08f7980595f0ddf664ff04f7e3c69d048b10fca1c04c115e4fee7d2e198: Status 404 returned error can't find the container with id 9836e08f7980595f0ddf664ff04f7e3c69d048b10fca1c04c115e4fee7d2e198 Jul 20 19:34:54.448481 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:34:54.448376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" event={"ID":"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a","Type":"ContainerStarted","Data":"9836e08f7980595f0ddf664ff04f7e3c69d048b10fca1c04c115e4fee7d2e198"} Jul 20 19:35:00.475324 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:00.475240 2580 generic.go:356] "Generic (PLEG): container finished" podID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerID="402780e0e428488ce03e838eba3aa11ff6f0d18e83e1c78f5c687eb0426dded0" exitCode=0 Jul 20 19:35:00.475324 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:00.475303 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" event={"ID":"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a","Type":"ContainerDied","Data":"402780e0e428488ce03e838eba3aa11ff6f0d18e83e1c78f5c687eb0426dded0"} Jul 20 19:35:03.488012 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:03.487974 2580 generic.go:356] "Generic (PLEG): container finished" podID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerID="0c791bb2aec3537a56a7ddbe28d8ce6e2bc60282b466507a5e3a15c017b995ca" exitCode=0 Jul 20 19:35:03.488411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:03.488018 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" event={"ID":"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a","Type":"ContainerDied","Data":"0c791bb2aec3537a56a7ddbe28d8ce6e2bc60282b466507a5e3a15c017b995ca"} Jul 20 19:35:11.519744 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:11.519711 2580 generic.go:356] "Generic (PLEG): container finished" podID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerID="021fcdda59bb99d619ae3b9d8503844a5240481b7e0b11e310a97dd7440415a3" exitCode=0 Jul 20 19:35:11.520178 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:11.519786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" event={"ID":"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a","Type":"ContainerDied","Data":"021fcdda59bb99d619ae3b9d8503844a5240481b7e0b11e310a97dd7440415a3"} Jul 20 19:35:12.688836 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.688800 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:35:12.785106 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.785086 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-util\") pod \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " Jul 20 19:35:12.785245 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.785128 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-bundle\") pod \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " Jul 20 19:35:12.785245 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.785201 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jnwr\" (UniqueName: \"kubernetes.io/projected/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-kube-api-access-2jnwr\") pod \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\" (UID: \"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a\") " Jul 20 19:35:12.785791 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.785766 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-bundle" (OuterVolumeSpecName: "bundle") pod "487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" (UID: "487a54ea-85e1-40b0-b57b-6e6ac7b30d7a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:35:12.787382 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.787332 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-kube-api-access-2jnwr" (OuterVolumeSpecName: "kube-api-access-2jnwr") pod "487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" (UID: "487a54ea-85e1-40b0-b57b-6e6ac7b30d7a"). InnerVolumeSpecName "kube-api-access-2jnwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:35:12.789625 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.789604 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-util" (OuterVolumeSpecName: "util") pod "487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" (UID: "487a54ea-85e1-40b0-b57b-6e6ac7b30d7a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:35:12.885737 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.885702 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jnwr\" (UniqueName: \"kubernetes.io/projected/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-kube-api-access-2jnwr\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:12.885737 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.885730 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-util\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:12.885737 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:12.885741 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/487a54ea-85e1-40b0-b57b-6e6ac7b30d7a-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:13.528451 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:13.528356 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" event={"ID":"487a54ea-85e1-40b0-b57b-6e6ac7b30d7a","Type":"ContainerDied","Data":"9836e08f7980595f0ddf664ff04f7e3c69d048b10fca1c04c115e4fee7d2e198"} Jul 20 19:35:13.528451 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:13.528384 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406kv2r" Jul 20 19:35:13.528451 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:13.528407 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9836e08f7980595f0ddf664ff04f7e3c69d048b10fca1c04c115e4fee7d2e198" Jul 20 19:35:16.787649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.787612 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s"] Jul 20 19:35:16.788096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.787876 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerName="pull" Jul 20 19:35:16.788096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.787899 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerName="pull" Jul 20 19:35:16.788096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.787915 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerName="extract" Jul 20 19:35:16.788096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.787923 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerName="extract" Jul 20 19:35:16.788096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.787947 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerName="util" Jul 20 19:35:16.788096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.787980 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerName="util" Jul 20 19:35:16.788096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.788048 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="487a54ea-85e1-40b0-b57b-6e6ac7b30d7a" containerName="extract" Jul 20 19:35:16.790577 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.790562 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:16.792689 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.792666 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:35:16.792689 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.792678 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 20 19:35:16.792866 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.792736 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c4qx2\"" Jul 20 19:35:16.803068 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.803046 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s"] Jul 20 19:35:16.919819 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.919786 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aecca74a-24a5-4d12-8f84-cb0f80e29bf9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wm95s\" (UID: \"aecca74a-24a5-4d12-8f84-cb0f80e29bf9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:16.920007 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:16.919840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5sf8\" (UniqueName: \"kubernetes.io/projected/aecca74a-24a5-4d12-8f84-cb0f80e29bf9-kube-api-access-w5sf8\") pod \"cert-manager-operator-controller-manager-55c444bf86-wm95s\" (UID: \"aecca74a-24a5-4d12-8f84-cb0f80e29bf9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:17.020999 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:17.020966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5sf8\" (UniqueName: \"kubernetes.io/projected/aecca74a-24a5-4d12-8f84-cb0f80e29bf9-kube-api-access-w5sf8\") pod \"cert-manager-operator-controller-manager-55c444bf86-wm95s\" (UID: \"aecca74a-24a5-4d12-8f84-cb0f80e29bf9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:17.021148 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:17.021025 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aecca74a-24a5-4d12-8f84-cb0f80e29bf9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wm95s\" (UID: \"aecca74a-24a5-4d12-8f84-cb0f80e29bf9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:17.021322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:17.021308 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aecca74a-24a5-4d12-8f84-cb0f80e29bf9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wm95s\" (UID: \"aecca74a-24a5-4d12-8f84-cb0f80e29bf9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:17.037280 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:17.037251 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5sf8\" (UniqueName: \"kubernetes.io/projected/aecca74a-24a5-4d12-8f84-cb0f80e29bf9-kube-api-access-w5sf8\") pod \"cert-manager-operator-controller-manager-55c444bf86-wm95s\" (UID: \"aecca74a-24a5-4d12-8f84-cb0f80e29bf9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:17.099667 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:17.099604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" Jul 20 19:35:17.231478 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:17.231452 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s"] Jul 20 19:35:17.234379 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:35:17.234350 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaecca74a_24a5_4d12_8f84_cb0f80e29bf9.slice/crio-62916e04322ef5b864429dd4dd3925a72ee3419421b71452eec95cb449689cb9 WatchSource:0}: Error finding container 62916e04322ef5b864429dd4dd3925a72ee3419421b71452eec95cb449689cb9: Status 404 returned error can't find the container with id 62916e04322ef5b864429dd4dd3925a72ee3419421b71452eec95cb449689cb9 Jul 20 19:35:17.542537 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:17.542508 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" event={"ID":"aecca74a-24a5-4d12-8f84-cb0f80e29bf9","Type":"ContainerStarted","Data":"62916e04322ef5b864429dd4dd3925a72ee3419421b71452eec95cb449689cb9"} Jul 20 19:35:19.553563 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:19.553532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" event={"ID":"aecca74a-24a5-4d12-8f84-cb0f80e29bf9","Type":"ContainerStarted","Data":"f94dd28b127ce452de3d1e15eb9151a88854e09bfca33c69fa837be97d40b429"} Jul 20 19:35:19.573146 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:19.573081 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wm95s" podStartSLOduration=1.926862589 podStartE2EDuration="3.573043299s" totalImagesPullingTime="1.64618071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:35:19.571108805 +0000 UTC m=+461.196187799" watchObservedRunningTime="2026-07-20 19:35:19.573043299 +0000 UTC m=+461.198122294" Jul 20 19:35:20.663322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.663288 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl"] Jul 20 19:35:20.665419 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.665403 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.667304 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.667274 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 19:35:20.667454 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.667371 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 19:35:20.667454 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.667419 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s2256\"" Jul 20 19:35:20.675937 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.675873 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl"] Jul 20 19:35:20.752198 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.752168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.752387 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.752270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.752387 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.752304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jv5j\" (UniqueName: \"kubernetes.io/projected/2439e890-6e5e-4254-81d1-20e96c91b173-kube-api-access-5jv5j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.852856 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.852814 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.852856 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.852862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jv5j\" (UniqueName: \"kubernetes.io/projected/2439e890-6e5e-4254-81d1-20e96c91b173-kube-api-access-5jv5j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.853040 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.852887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.853258 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.853243 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.853293 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.853240 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.865767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.865735 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jv5j\" (UniqueName: \"kubernetes.io/projected/2439e890-6e5e-4254-81d1-20e96c91b173-kube-api-access-5jv5j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:20.974786 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:20.974699 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:21.118175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:21.118143 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl"] Jul 20 19:35:21.561992 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:21.561954 2580 generic.go:356] "Generic (PLEG): container finished" podID="2439e890-6e5e-4254-81d1-20e96c91b173" containerID="da6d0610b30a22675cab40efd1f228d136c648ff5ac08cd151431c0f66369736" exitCode=0 Jul 20 19:35:21.562174 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:21.562049 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" event={"ID":"2439e890-6e5e-4254-81d1-20e96c91b173","Type":"ContainerDied","Data":"da6d0610b30a22675cab40efd1f228d136c648ff5ac08cd151431c0f66369736"} Jul 20 19:35:21.562174 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:21.562136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" event={"ID":"2439e890-6e5e-4254-81d1-20e96c91b173","Type":"ContainerStarted","Data":"531a1351649c3860223d57c0286494632fc373d712093454b5c4ea70d61faf5f"} Jul 20 19:35:24.576234 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:24.576196 2580 generic.go:356] "Generic (PLEG): container finished" podID="2439e890-6e5e-4254-81d1-20e96c91b173" containerID="4a49a016868ad84fa13f2faae4c6f09efd94bd22f1d4f68c313fda0681e2558f" exitCode=0 Jul 20 19:35:24.576674 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:24.576282 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" event={"ID":"2439e890-6e5e-4254-81d1-20e96c91b173","Type":"ContainerDied","Data":"4a49a016868ad84fa13f2faae4c6f09efd94bd22f1d4f68c313fda0681e2558f"} Jul 20 19:35:25.582208 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:25.582175 2580 generic.go:356] "Generic (PLEG): container finished" podID="2439e890-6e5e-4254-81d1-20e96c91b173" containerID="7c18fe5b785f372e40ba76e27111de467f1fa62fd9319f4fd1f6c198dc37d3ea" exitCode=0 Jul 20 19:35:25.582611 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:25.582231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" event={"ID":"2439e890-6e5e-4254-81d1-20e96c91b173","Type":"ContainerDied","Data":"7c18fe5b785f372e40ba76e27111de467f1fa62fd9319f4fd1f6c198dc37d3ea"} Jul 20 19:35:26.708812 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.708789 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:26.808097 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.808062 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jv5j\" (UniqueName: \"kubernetes.io/projected/2439e890-6e5e-4254-81d1-20e96c91b173-kube-api-access-5jv5j\") pod \"2439e890-6e5e-4254-81d1-20e96c91b173\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " Jul 20 19:35:26.808275 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.808113 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-bundle\") pod \"2439e890-6e5e-4254-81d1-20e96c91b173\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " Jul 20 19:35:26.808275 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.808134 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-util\") pod \"2439e890-6e5e-4254-81d1-20e96c91b173\" (UID: \"2439e890-6e5e-4254-81d1-20e96c91b173\") " Jul 20 19:35:26.808544 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.808516 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-bundle" (OuterVolumeSpecName: "bundle") pod "2439e890-6e5e-4254-81d1-20e96c91b173" (UID: "2439e890-6e5e-4254-81d1-20e96c91b173"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:35:26.810204 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.810171 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2439e890-6e5e-4254-81d1-20e96c91b173-kube-api-access-5jv5j" (OuterVolumeSpecName: "kube-api-access-5jv5j") pod "2439e890-6e5e-4254-81d1-20e96c91b173" (UID: "2439e890-6e5e-4254-81d1-20e96c91b173"). InnerVolumeSpecName "kube-api-access-5jv5j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:35:26.812951 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.812929 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-util" (OuterVolumeSpecName: "util") pod "2439e890-6e5e-4254-81d1-20e96c91b173" (UID: "2439e890-6e5e-4254-81d1-20e96c91b173"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:35:26.908787 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.908690 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jv5j\" (UniqueName: \"kubernetes.io/projected/2439e890-6e5e-4254-81d1-20e96c91b173-kube-api-access-5jv5j\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:26.908787 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.908733 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:26.908787 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:26.908742 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2439e890-6e5e-4254-81d1-20e96c91b173-util\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:27.592046 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:27.592008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" event={"ID":"2439e890-6e5e-4254-81d1-20e96c91b173","Type":"ContainerDied","Data":"531a1351649c3860223d57c0286494632fc373d712093454b5c4ea70d61faf5f"} Jul 20 19:35:27.592287 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:27.592055 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="531a1351649c3860223d57c0286494632fc373d712093454b5c4ea70d61faf5f" Jul 20 19:35:27.592287 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:27.592023 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fntjsl" Jul 20 19:35:31.070296 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070264 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74"] Jul 20 19:35:31.070690 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070531 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2439e890-6e5e-4254-81d1-20e96c91b173" containerName="pull" Jul 20 19:35:31.070690 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070544 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2439e890-6e5e-4254-81d1-20e96c91b173" containerName="pull" Jul 20 19:35:31.070690 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070558 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2439e890-6e5e-4254-81d1-20e96c91b173" containerName="extract" Jul 20 19:35:31.070690 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070563 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2439e890-6e5e-4254-81d1-20e96c91b173" containerName="extract" Jul 20 19:35:31.070690 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070571 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2439e890-6e5e-4254-81d1-20e96c91b173" containerName="util" Jul 20 19:35:31.070690 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070576 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2439e890-6e5e-4254-81d1-20e96c91b173" containerName="util" Jul 20 19:35:31.070690 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.070618 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2439e890-6e5e-4254-81d1-20e96c91b173" containerName="extract" Jul 20 19:35:31.073534 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.073516 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.075510 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.075494 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:35:31.075938 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.075920 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mdkxm\"" Jul 20 19:35:31.075997 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.075955 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 19:35:31.082797 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.082775 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74"] Jul 20 19:35:31.141022 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.140983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2sf5\" (UniqueName: \"kubernetes.io/projected/86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e-kube-api-access-s2sf5\") pod \"openshift-lws-operator-fd8ccff4c-wsq74\" (UID: \"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.141198 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.141083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wsq74\" (UID: \"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.242296 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.242260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wsq74\" (UID: \"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.242492 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.242319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2sf5\" (UniqueName: \"kubernetes.io/projected/86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e-kube-api-access-s2sf5\") pod \"openshift-lws-operator-fd8ccff4c-wsq74\" (UID: \"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.242753 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.242732 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wsq74\" (UID: \"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.250284 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.250260 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2sf5\" (UniqueName: \"kubernetes.io/projected/86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e-kube-api-access-s2sf5\") pod \"openshift-lws-operator-fd8ccff4c-wsq74\" (UID: \"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.382237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.382135 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" Jul 20 19:35:31.508473 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.508449 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74"] Jul 20 19:35:31.511278 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:35:31.511245 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86eb4aaf_d9de_4f34_91e1_0404b3d4cc1e.slice/crio-923e6bbcaf4d4a239ace4e936434818971dc2e52df5d1449071e6a91ae6d404a WatchSource:0}: Error finding container 923e6bbcaf4d4a239ace4e936434818971dc2e52df5d1449071e6a91ae6d404a: Status 404 returned error can't find the container with id 923e6bbcaf4d4a239ace4e936434818971dc2e52df5d1449071e6a91ae6d404a Jul 20 19:35:31.546726 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.546690 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9jlm6"] Jul 20 19:35:31.549607 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.549587 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.551467 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.551447 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 19:35:31.551588 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.551571 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7587s\"" Jul 20 19:35:31.551648 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.551635 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 19:35:31.559507 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.559482 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9jlm6"] Jul 20 19:35:31.609046 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.609009 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" event={"ID":"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e","Type":"ContainerStarted","Data":"923e6bbcaf4d4a239ace4e936434818971dc2e52df5d1449071e6a91ae6d404a"} Jul 20 19:35:31.645487 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.645398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4c1da90-b53c-4144-873a-b06bdab3c785-bound-sa-token\") pod \"cert-manager-55b4d954d5-9jlm6\" (UID: \"a4c1da90-b53c-4144-873a-b06bdab3c785\") " pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.645657 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.645562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg6j9\" (UniqueName: \"kubernetes.io/projected/a4c1da90-b53c-4144-873a-b06bdab3c785-kube-api-access-cg6j9\") pod \"cert-manager-55b4d954d5-9jlm6\" (UID: \"a4c1da90-b53c-4144-873a-b06bdab3c785\") " pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.746727 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.746688 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg6j9\" (UniqueName: \"kubernetes.io/projected/a4c1da90-b53c-4144-873a-b06bdab3c785-kube-api-access-cg6j9\") pod \"cert-manager-55b4d954d5-9jlm6\" (UID: \"a4c1da90-b53c-4144-873a-b06bdab3c785\") " pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.746727 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.746729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4c1da90-b53c-4144-873a-b06bdab3c785-bound-sa-token\") pod \"cert-manager-55b4d954d5-9jlm6\" (UID: \"a4c1da90-b53c-4144-873a-b06bdab3c785\") " pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.754026 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.754002 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4c1da90-b53c-4144-873a-b06bdab3c785-bound-sa-token\") pod \"cert-manager-55b4d954d5-9jlm6\" (UID: \"a4c1da90-b53c-4144-873a-b06bdab3c785\") " pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.754146 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.754126 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg6j9\" (UniqueName: \"kubernetes.io/projected/a4c1da90-b53c-4144-873a-b06bdab3c785-kube-api-access-cg6j9\") pod \"cert-manager-55b4d954d5-9jlm6\" (UID: \"a4c1da90-b53c-4144-873a-b06bdab3c785\") " pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.859496 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.859461 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9jlm6" Jul 20 19:35:31.981110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:31.981074 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9jlm6"] Jul 20 19:35:31.983670 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:35:31.983639 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4c1da90_b53c_4144_873a_b06bdab3c785.slice/crio-eb0b8755e6eda5398bbc487fd004179f20203374b1a8a68ac71c994b63363f90 WatchSource:0}: Error finding container eb0b8755e6eda5398bbc487fd004179f20203374b1a8a68ac71c994b63363f90: Status 404 returned error can't find the container with id eb0b8755e6eda5398bbc487fd004179f20203374b1a8a68ac71c994b63363f90 Jul 20 19:35:32.616104 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:32.616074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9jlm6" event={"ID":"a4c1da90-b53c-4144-873a-b06bdab3c785","Type":"ContainerStarted","Data":"eb0b8755e6eda5398bbc487fd004179f20203374b1a8a68ac71c994b63363f90"} Jul 20 19:35:34.628752 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:34.628708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" event={"ID":"86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e","Type":"ContainerStarted","Data":"c10d4b1bd69a431734b6e314ddf326146ea3bb0b110c67b04d3f860c6043a698"} Jul 20 19:35:34.643752 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:34.643509 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wsq74" podStartSLOduration=1.448420665 podStartE2EDuration="3.643490769s" totalImagesPullingTime="2.195070104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:35:34.642844598 +0000 UTC m=+476.267923594" watchObservedRunningTime="2026-07-20 19:35:34.643490769 +0000 UTC m=+476.268569763" Jul 20 19:35:35.633842 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:35.633804 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9jlm6" event={"ID":"a4c1da90-b53c-4144-873a-b06bdab3c785","Type":"ContainerStarted","Data":"498224300e0e14ab359872008de14c560752bc458b35e0450938a22ea9047c8c"} Jul 20 19:35:35.649528 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:35.649481 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9jlm6" podStartSLOduration=1.847292003 podStartE2EDuration="4.649466108s" totalImagesPullingTime="2.802174105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:35:35.647589108 +0000 UTC m=+477.272668116" watchObservedRunningTime="2026-07-20 19:35:35.649466108 +0000 UTC m=+477.274545102" Jul 20 19:35:37.624020 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.623985 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt"] Jul 20 19:35:37.626303 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.626286 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.628994 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.628973 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s2256\"" Jul 20 19:35:37.629274 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.629253 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 19:35:37.629354 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.629319 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 19:35:37.637349 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.637325 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt"] Jul 20 19:35:37.699096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.699053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.699096 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.699091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.699302 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.699212 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2kvh\" (UniqueName: \"kubernetes.io/projected/5bd88ab6-8bd0-4663-8e58-3182962dac8e-kube-api-access-z2kvh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.800440 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.800398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.800604 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.800469 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.800604 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.800566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2kvh\" (UniqueName: \"kubernetes.io/projected/5bd88ab6-8bd0-4663-8e58-3182962dac8e-kube-api-access-z2kvh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.800865 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.800843 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.800903 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.800853 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.808641 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.808609 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2kvh\" (UniqueName: \"kubernetes.io/projected/5bd88ab6-8bd0-4663-8e58-3182962dac8e-kube-api-access-z2kvh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:37.935380 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:37.935284 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:38.064119 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:38.064093 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt"] Jul 20 19:35:38.066335 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:35:38.066307 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd88ab6_8bd0_4663_8e58_3182962dac8e.slice/crio-9e703384c04792add652df5b160c4afa9bb66c997bf33a5fd9283ccef2a1eec1 WatchSource:0}: Error finding container 9e703384c04792add652df5b160c4afa9bb66c997bf33a5fd9283ccef2a1eec1: Status 404 returned error can't find the container with id 9e703384c04792add652df5b160c4afa9bb66c997bf33a5fd9283ccef2a1eec1 Jul 20 19:35:38.647803 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:38.647764 2580 generic.go:356] "Generic (PLEG): container finished" podID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerID="f500f25c65ed017094e794bbea88a5f55532bea66df9a0a80c3cb1e4cc2ae216" exitCode=0 Jul 20 19:35:38.648227 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:38.647851 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" event={"ID":"5bd88ab6-8bd0-4663-8e58-3182962dac8e","Type":"ContainerDied","Data":"f500f25c65ed017094e794bbea88a5f55532bea66df9a0a80c3cb1e4cc2ae216"} Jul 20 19:35:38.648227 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:38.647907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" event={"ID":"5bd88ab6-8bd0-4663-8e58-3182962dac8e","Type":"ContainerStarted","Data":"9e703384c04792add652df5b160c4afa9bb66c997bf33a5fd9283ccef2a1eec1"} Jul 20 19:35:39.653660 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:39.653626 2580 generic.go:356] "Generic (PLEG): container finished" podID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerID="769bcdc960310dd05f8beff659115c3b64aad56a86456be2dd653c4bca016bbd" exitCode=0 Jul 20 19:35:39.654124 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:39.653701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" event={"ID":"5bd88ab6-8bd0-4663-8e58-3182962dac8e","Type":"ContainerDied","Data":"769bcdc960310dd05f8beff659115c3b64aad56a86456be2dd653c4bca016bbd"} Jul 20 19:35:40.661041 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:40.661002 2580 generic.go:356] "Generic (PLEG): container finished" podID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerID="db187cea9131859bf479b7ecacf22980ca01a0d103db87b69c30c18b455c550c" exitCode=0 Jul 20 19:35:40.661519 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:40.661089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" event={"ID":"5bd88ab6-8bd0-4663-8e58-3182962dac8e","Type":"ContainerDied","Data":"db187cea9131859bf479b7ecacf22980ca01a0d103db87b69c30c18b455c550c"} Jul 20 19:35:41.837976 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:41.837945 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:41.932994 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:41.932907 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2kvh\" (UniqueName: \"kubernetes.io/projected/5bd88ab6-8bd0-4663-8e58-3182962dac8e-kube-api-access-z2kvh\") pod \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " Jul 20 19:35:41.932994 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:41.932970 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-bundle\") pod \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " Jul 20 19:35:41.932994 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:41.932987 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-util\") pod \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\" (UID: \"5bd88ab6-8bd0-4663-8e58-3182962dac8e\") " Jul 20 19:35:41.933706 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:41.933679 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-bundle" (OuterVolumeSpecName: "bundle") pod "5bd88ab6-8bd0-4663-8e58-3182962dac8e" (UID: "5bd88ab6-8bd0-4663-8e58-3182962dac8e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:35:41.934936 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:41.934911 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bd88ab6-8bd0-4663-8e58-3182962dac8e-kube-api-access-z2kvh" (OuterVolumeSpecName: "kube-api-access-z2kvh") pod "5bd88ab6-8bd0-4663-8e58-3182962dac8e" (UID: "5bd88ab6-8bd0-4663-8e58-3182962dac8e"). InnerVolumeSpecName "kube-api-access-z2kvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:35:41.941137 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:41.941118 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-util" (OuterVolumeSpecName: "util") pod "5bd88ab6-8bd0-4663-8e58-3182962dac8e" (UID: "5bd88ab6-8bd0-4663-8e58-3182962dac8e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:35:42.034031 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:42.033996 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2kvh\" (UniqueName: \"kubernetes.io/projected/5bd88ab6-8bd0-4663-8e58-3182962dac8e-kube-api-access-z2kvh\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:42.034031 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:42.034026 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:42.034031 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:42.034035 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bd88ab6-8bd0-4663-8e58-3182962dac8e-util\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:35:42.671825 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:42.671792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" event={"ID":"5bd88ab6-8bd0-4663-8e58-3182962dac8e","Type":"ContainerDied","Data":"9e703384c04792add652df5b160c4afa9bb66c997bf33a5fd9283ccef2a1eec1"} Jul 20 19:35:42.671825 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:42.671823 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af42xnt" Jul 20 19:35:42.672051 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:42.671837 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e703384c04792add652df5b160c4afa9bb66c997bf33a5fd9283ccef2a1eec1" Jul 20 19:35:50.515412 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515377 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj"] Jul 20 19:35:50.515972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515687 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerName="pull" Jul 20 19:35:50.515972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515698 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerName="pull" Jul 20 19:35:50.515972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515711 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerName="util" Jul 20 19:35:50.515972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515717 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerName="util" Jul 20 19:35:50.515972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515728 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerName="extract" Jul 20 19:35:50.515972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515733 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerName="extract" Jul 20 19:35:50.515972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.515792 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd88ab6-8bd0-4663-8e58-3182962dac8e" containerName="extract" Jul 20 19:35:50.518487 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.518470 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.521242 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.521222 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 19:35:50.521570 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.521546 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 19:35:50.521678 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.521605 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pg8q9\"" Jul 20 19:35:50.521678 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.521618 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 19:35:50.521678 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.521652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 19:35:50.542493 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.542463 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj"] Jul 20 19:35:50.602961 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.602927 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjm9k\" (UniqueName: \"kubernetes.io/projected/7ac466b5-52bf-4284-808d-7281f3e59775-kube-api-access-vjm9k\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.603146 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.602984 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7ac466b5-52bf-4284-808d-7281f3e59775-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.603146 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.603013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ac466b5-52bf-4284-808d-7281f3e59775-tmp\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.603146 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.603049 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7ac466b5-52bf-4284-808d-7281f3e59775-manifests\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.603146 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.603073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ac466b5-52bf-4284-808d-7281f3e59775-webhook-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.703602 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.703561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7ac466b5-52bf-4284-808d-7281f3e59775-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.703602 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.703605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ac466b5-52bf-4284-808d-7281f3e59775-tmp\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.703855 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.703658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7ac466b5-52bf-4284-808d-7281f3e59775-manifests\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.703855 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.703681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ac466b5-52bf-4284-808d-7281f3e59775-webhook-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.703855 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.703728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjm9k\" (UniqueName: \"kubernetes.io/projected/7ac466b5-52bf-4284-808d-7281f3e59775-kube-api-access-vjm9k\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.704185 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.704152 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7ac466b5-52bf-4284-808d-7281f3e59775-manifests\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.706253 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.706228 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7ac466b5-52bf-4284-808d-7281f3e59775-webhook-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.706368 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.706319 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ac466b5-52bf-4284-808d-7281f3e59775-tmp\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.706693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.706670 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7ac466b5-52bf-4284-808d-7281f3e59775-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.711254 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.711234 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjm9k\" (UniqueName: \"kubernetes.io/projected/7ac466b5-52bf-4284-808d-7281f3e59775-kube-api-access-vjm9k\") pod \"opendatahub-operator-controller-manager-859c78d9fd-v4vzj\" (UID: \"7ac466b5-52bf-4284-808d-7281f3e59775\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.828605 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.828568 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:50.969373 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:50.969345 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj"] Jul 20 19:35:50.973339 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:35:50.973308 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac466b5_52bf_4284_808d_7281f3e59775.slice/crio-b48a48c674196b17a8d40b33ec9c38a35e612ba3c2364711fe1531ec968e34b6 WatchSource:0}: Error finding container b48a48c674196b17a8d40b33ec9c38a35e612ba3c2364711fe1531ec968e34b6: Status 404 returned error can't find the container with id b48a48c674196b17a8d40b33ec9c38a35e612ba3c2364711fe1531ec968e34b6 Jul 20 19:35:51.714687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:51.714607 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" event={"ID":"7ac466b5-52bf-4284-808d-7281f3e59775","Type":"ContainerStarted","Data":"b48a48c674196b17a8d40b33ec9c38a35e612ba3c2364711fe1531ec968e34b6"} Jul 20 19:35:54.730222 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:54.730176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" event={"ID":"7ac466b5-52bf-4284-808d-7281f3e59775","Type":"ContainerStarted","Data":"cda85436f2a8cea7ce7e1fdf182c42fcabf89975d42429cc9f36833a3e61d39b"} Jul 20 19:35:55.736618 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:55.736574 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" event={"ID":"7ac466b5-52bf-4284-808d-7281f3e59775","Type":"ContainerDied","Data":"cda85436f2a8cea7ce7e1fdf182c42fcabf89975d42429cc9f36833a3e61d39b"} Jul 20 19:35:55.737076 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:55.736497 2580 generic.go:356] "Generic (PLEG): container finished" podID="7ac466b5-52bf-4284-808d-7281f3e59775" containerID="cda85436f2a8cea7ce7e1fdf182c42fcabf89975d42429cc9f36833a3e61d39b" exitCode=0 Jul 20 19:35:56.742892 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:56.742851 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" event={"ID":"7ac466b5-52bf-4284-808d-7281f3e59775","Type":"ContainerStarted","Data":"f3c94f8bc81a0f2eaf9c622307ce51c8da34af5d34d2b2e5f59bb9d14627b189"} Jul 20 19:35:56.743301 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:56.742942 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:35:56.766094 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:35:56.766047 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" podStartSLOduration=2.375573473 podStartE2EDuration="6.766031648s" totalImagesPullingTime="3.300491997s" totalInitContainerRuntime="1.089966178s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:35:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:35:56.7654869 +0000 UTC m=+498.390565895" watchObservedRunningTime="2026-07-20 19:35:56.766031648 +0000 UTC m=+498.391110641" Jul 20 19:36:07.749707 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:07.749676 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-v4vzj" Jul 20 19:36:09.976589 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:09.976549 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gxzbw"] Jul 20 19:36:09.979781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:09.979761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:09.981831 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:09.981810 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cbc72\"" Jul 20 19:36:09.983160 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:09.983143 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 20 19:36:09.997394 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:09.997371 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gxzbw"] Jul 20 19:36:10.064900 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.064865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxdvn\" (UniqueName: \"kubernetes.io/projected/871baf92-5f83-4e69-a5af-6aba4bb234d3-kube-api-access-lxdvn\") pod \"odh-model-controller-7998c67dc4-gxzbw\" (UID: \"871baf92-5f83-4e69-a5af-6aba4bb234d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:10.065065 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.064937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/871baf92-5f83-4e69-a5af-6aba4bb234d3-cert\") pod \"odh-model-controller-7998c67dc4-gxzbw\" (UID: \"871baf92-5f83-4e69-a5af-6aba4bb234d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:10.165638 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.165606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxdvn\" (UniqueName: \"kubernetes.io/projected/871baf92-5f83-4e69-a5af-6aba4bb234d3-kube-api-access-lxdvn\") pod \"odh-model-controller-7998c67dc4-gxzbw\" (UID: \"871baf92-5f83-4e69-a5af-6aba4bb234d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:10.165831 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.165657 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/871baf92-5f83-4e69-a5af-6aba4bb234d3-cert\") pod \"odh-model-controller-7998c67dc4-gxzbw\" (UID: \"871baf92-5f83-4e69-a5af-6aba4bb234d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:10.168138 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.168113 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/871baf92-5f83-4e69-a5af-6aba4bb234d3-cert\") pod \"odh-model-controller-7998c67dc4-gxzbw\" (UID: \"871baf92-5f83-4e69-a5af-6aba4bb234d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:10.180243 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.180220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxdvn\" (UniqueName: \"kubernetes.io/projected/871baf92-5f83-4e69-a5af-6aba4bb234d3-kube-api-access-lxdvn\") pod \"odh-model-controller-7998c67dc4-gxzbw\" (UID: \"871baf92-5f83-4e69-a5af-6aba4bb234d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:10.289287 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.289260 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:10.416165 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.416141 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gxzbw"] Jul 20 19:36:10.418398 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:36:10.418367 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871baf92_5f83_4e69_a5af_6aba4bb234d3.slice/crio-531e2636a48211348641e4209c628d5253aea1041e0fbeb445c843a904fb27f3 WatchSource:0}: Error finding container 531e2636a48211348641e4209c628d5253aea1041e0fbeb445c843a904fb27f3: Status 404 returned error can't find the container with id 531e2636a48211348641e4209c628d5253aea1041e0fbeb445c843a904fb27f3 Jul 20 19:36:10.608736 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.608656 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hz8g8"] Jul 20 19:36:10.612882 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.612858 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:10.614645 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.614627 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 20 19:36:10.614744 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.614657 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2hx6l\"" Jul 20 19:36:10.620099 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.620068 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hz8g8"] Jul 20 19:36:10.770365 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.770333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b46b79d3-e102-409f-ad51-3e3376954fc6-tls-certs\") pod \"model-serving-api-54d89bd5fc-hz8g8\" (UID: \"b46b79d3-e102-409f-ad51-3e3376954fc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:10.770594 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.770412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjxt7\" (UniqueName: \"kubernetes.io/projected/b46b79d3-e102-409f-ad51-3e3376954fc6-kube-api-access-vjxt7\") pod \"model-serving-api-54d89bd5fc-hz8g8\" (UID: \"b46b79d3-e102-409f-ad51-3e3376954fc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:10.806038 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.806004 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" event={"ID":"871baf92-5f83-4e69-a5af-6aba4bb234d3","Type":"ContainerStarted","Data":"531e2636a48211348641e4209c628d5253aea1041e0fbeb445c843a904fb27f3"} Jul 20 19:36:10.871311 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.871239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjxt7\" (UniqueName: \"kubernetes.io/projected/b46b79d3-e102-409f-ad51-3e3376954fc6-kube-api-access-vjxt7\") pod \"model-serving-api-54d89bd5fc-hz8g8\" (UID: \"b46b79d3-e102-409f-ad51-3e3376954fc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:10.871311 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.871284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b46b79d3-e102-409f-ad51-3e3376954fc6-tls-certs\") pod \"model-serving-api-54d89bd5fc-hz8g8\" (UID: \"b46b79d3-e102-409f-ad51-3e3376954fc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:10.873639 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.873621 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b46b79d3-e102-409f-ad51-3e3376954fc6-tls-certs\") pod \"model-serving-api-54d89bd5fc-hz8g8\" (UID: \"b46b79d3-e102-409f-ad51-3e3376954fc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:10.879062 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.879040 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjxt7\" (UniqueName: \"kubernetes.io/projected/b46b79d3-e102-409f-ad51-3e3376954fc6-kube-api-access-vjxt7\") pod \"model-serving-api-54d89bd5fc-hz8g8\" (UID: \"b46b79d3-e102-409f-ad51-3e3376954fc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:10.924609 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:10.924583 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:11.053157 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:11.053130 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hz8g8"] Jul 20 19:36:11.054516 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:36:11.054490 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb46b79d3_e102_409f_ad51_3e3376954fc6.slice/crio-6dac54865b2541fd5137e60182cb24595869312c0052db0ce12d3c712fc2ecf7 WatchSource:0}: Error finding container 6dac54865b2541fd5137e60182cb24595869312c0052db0ce12d3c712fc2ecf7: Status 404 returned error can't find the container with id 6dac54865b2541fd5137e60182cb24595869312c0052db0ce12d3c712fc2ecf7 Jul 20 19:36:11.814157 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:11.814110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" event={"ID":"b46b79d3-e102-409f-ad51-3e3376954fc6","Type":"ContainerStarted","Data":"6dac54865b2541fd5137e60182cb24595869312c0052db0ce12d3c712fc2ecf7"} Jul 20 19:36:13.825317 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:13.825228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" event={"ID":"871baf92-5f83-4e69-a5af-6aba4bb234d3","Type":"ContainerStarted","Data":"3415c5f359b2a4dacf666f3c30acbb12793badc4810ab8c3b837b9445f70ce68"} Jul 20 19:36:13.825796 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:13.825368 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:13.826923 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:13.826880 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" event={"ID":"b46b79d3-e102-409f-ad51-3e3376954fc6","Type":"ContainerStarted","Data":"1dbeb1a12c25f224e7df82b2dae76db610b5f396931c29c6cd7bf804b77d5ac6"} Jul 20 19:36:13.827033 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:13.826996 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:13.846939 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:13.846897 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" podStartSLOduration=1.709607345 podStartE2EDuration="4.846883732s" totalImagesPullingTime="3.137276387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:13.844254701 +0000 UTC m=+515.469333695" watchObservedRunningTime="2026-07-20 19:36:13.846883732 +0000 UTC m=+515.471962727" Jul 20 19:36:13.859529 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:13.859481 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" podStartSLOduration=1.361932162 podStartE2EDuration="3.859462705s" totalImagesPullingTime="2.497530543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:13.859018843 +0000 UTC m=+515.484097837" watchObservedRunningTime="2026-07-20 19:36:13.859462705 +0000 UTC m=+515.484541699" Jul 20 19:36:16.905766 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.905733 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm"] Jul 20 19:36:16.909021 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.908999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:16.910922 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.910905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 19:36:16.910995 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.910941 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 20 19:36:16.911060 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.911042 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 20 19:36:16.911172 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.911148 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-t67g2\"" Jul 20 19:36:16.911298 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.911286 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 20 19:36:16.931650 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:16.931620 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm"] Jul 20 19:36:17.037782 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.037739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7nf5\" (UniqueName: \"kubernetes.io/projected/40dcbb61-bb8e-4084-b820-4719b106c922-kube-api-access-h7nf5\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.037972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.037791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.037972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.037824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40dcbb61-bb8e-4084-b820-4719b106c922-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.037972 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.037949 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dcbb61-bb8e-4084-b820-4719b106c922-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.038110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.037991 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.038110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.038044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.038110 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.038067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dcbb61-bb8e-4084-b820-4719b106c922-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.138515 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.138479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.138694 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.138529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dcbb61-bb8e-4084-b820-4719b106c922-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.138793 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.138729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7nf5\" (UniqueName: \"kubernetes.io/projected/40dcbb61-bb8e-4084-b820-4719b106c922-kube-api-access-h7nf5\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.138861 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.138784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.138861 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.138847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40dcbb61-bb8e-4084-b820-4719b106c922-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.139066 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.138947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dcbb61-bb8e-4084-b820-4719b106c922-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.139066 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.138983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.139643 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.139614 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40dcbb61-bb8e-4084-b820-4719b106c922-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.141312 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.141286 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dcbb61-bb8e-4084-b820-4719b106c922-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.141673 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.141651 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.141757 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.141673 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.141757 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.141706 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40dcbb61-bb8e-4084-b820-4719b106c922-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.153874 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.153855 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dcbb61-bb8e-4084-b820-4719b106c922-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.157632 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.157579 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7nf5\" (UniqueName: \"kubernetes.io/projected/40dcbb61-bb8e-4084-b820-4719b106c922-kube-api-access-h7nf5\") pod \"istiod-openshift-gateway-94bb8fbfd-ppscm\" (UID: \"40dcbb61-bb8e-4084-b820-4719b106c922\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.217557 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.217530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:17.357793 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.357766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm"] Jul 20 19:36:17.359707 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:36:17.359676 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40dcbb61_bb8e_4084_b820_4719b106c922.slice/crio-51fe87c2400335cdb8c4d342fd28e81460d49ac16af460ea5e0fbafdf7da00bd WatchSource:0}: Error finding container 51fe87c2400335cdb8c4d342fd28e81460d49ac16af460ea5e0fbafdf7da00bd: Status 404 returned error can't find the container with id 51fe87c2400335cdb8c4d342fd28e81460d49ac16af460ea5e0fbafdf7da00bd Jul 20 19:36:17.845030 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:17.844993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" event={"ID":"40dcbb61-bb8e-4084-b820-4719b106c922","Type":"ContainerStarted","Data":"51fe87c2400335cdb8c4d342fd28e81460d49ac16af460ea5e0fbafdf7da00bd"} Jul 20 19:36:20.057347 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.057238 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8"] Jul 20 19:36:20.066413 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.066380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.069747 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.069412 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ppr9m\"" Jul 20 19:36:20.071644 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.071541 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 20 19:36:20.081600 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.080711 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj"] Jul 20 19:36:20.102457 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.101632 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8"] Jul 20 19:36:20.102457 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.101817 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.108448 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.108036 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj"] Jul 20 19:36:20.108448 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.108226 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-r62wn\"" Jul 20 19:36:20.110075 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.110055 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 20 19:36:20.167125 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.167089 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9442t\" (UniqueName: \"kubernetes.io/projected/375cbf6c-bbc2-46f6-8711-ea94b37ec6ff-kube-api-access-9442t\") pod \"kserve-controller-manager-d6f4b65b8-xjpq8\" (UID: \"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.167325 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.167138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/375cbf6c-bbc2-46f6-8711-ea94b37ec6ff-cert\") pod \"kserve-controller-manager-d6f4b65b8-xjpq8\" (UID: \"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.268287 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.268245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wl8\" (UniqueName: \"kubernetes.io/projected/c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7-kube-api-access-r9wl8\") pod \"llmisvc-controller-manager-777c6bd7d5-gbdwj\" (UID: \"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.268468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.268300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gbdwj\" (UID: \"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.268468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.268367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9442t\" (UniqueName: \"kubernetes.io/projected/375cbf6c-bbc2-46f6-8711-ea94b37ec6ff-kube-api-access-9442t\") pod \"kserve-controller-manager-d6f4b65b8-xjpq8\" (UID: \"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.268468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.268410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/375cbf6c-bbc2-46f6-8711-ea94b37ec6ff-cert\") pod \"kserve-controller-manager-d6f4b65b8-xjpq8\" (UID: \"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.270878 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.270849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/375cbf6c-bbc2-46f6-8711-ea94b37ec6ff-cert\") pod \"kserve-controller-manager-d6f4b65b8-xjpq8\" (UID: \"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.282211 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.282187 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9442t\" (UniqueName: \"kubernetes.io/projected/375cbf6c-bbc2-46f6-8711-ea94b37ec6ff-kube-api-access-9442t\") pod \"kserve-controller-manager-d6f4b65b8-xjpq8\" (UID: \"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.296332 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.296299 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:36:20.296409 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.296363 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:36:20.369214 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.369188 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wl8\" (UniqueName: \"kubernetes.io/projected/c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7-kube-api-access-r9wl8\") pod \"llmisvc-controller-manager-777c6bd7d5-gbdwj\" (UID: \"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.369325 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.369242 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gbdwj\" (UID: \"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.371370 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.371351 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gbdwj\" (UID: \"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.377741 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.377721 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wl8\" (UniqueName: \"kubernetes.io/projected/c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7-kube-api-access-r9wl8\") pod \"llmisvc-controller-manager-777c6bd7d5-gbdwj\" (UID: \"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.384518 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.384498 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:20.427904 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.427859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:20.535014 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.534981 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8"] Jul 20 19:36:20.804260 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.804229 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj"] Jul 20 19:36:20.806307 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:36:20.806276 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc02f3ea0_0f4e_42dd_9fe0_4ca4f3059aa7.slice/crio-297e013e161f69f2b2e54952b5acde3b8bafeaae3681eb3ebbc90964b83e13f7 WatchSource:0}: Error finding container 297e013e161f69f2b2e54952b5acde3b8bafeaae3681eb3ebbc90964b83e13f7: Status 404 returned error can't find the container with id 297e013e161f69f2b2e54952b5acde3b8bafeaae3681eb3ebbc90964b83e13f7 Jul 20 19:36:20.864859 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.864821 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" event={"ID":"40dcbb61-bb8e-4084-b820-4719b106c922","Type":"ContainerStarted","Data":"3c2ab6847e72fda73f430b7779e18ef1eda16d90c69c25e4eb75aadc274a03d7"} Jul 20 19:36:20.865039 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.864928 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:20.866114 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.866084 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" event={"ID":"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7","Type":"ContainerStarted","Data":"297e013e161f69f2b2e54952b5acde3b8bafeaae3681eb3ebbc90964b83e13f7"} Jul 20 19:36:20.867192 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.867169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" event={"ID":"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff","Type":"ContainerStarted","Data":"ee0cf9fef80f605f42d2336049f534abfd62d8d2385f4fa3a4a84a5797d3af28"} Jul 20 19:36:20.887389 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:20.887340 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" podStartSLOduration=1.9529472989999999 podStartE2EDuration="4.887323584s" totalImagesPullingTime="2.934376285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:20.886120888 +0000 UTC m=+522.511199907" watchObservedRunningTime="2026-07-20 19:36:20.887323584 +0000 UTC m=+522.512402579" Jul 20 19:36:21.875655 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:21.875620 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-ppscm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 19:36:21.876322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:21.875674 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" podUID="40dcbb61-bb8e-4084-b820-4719b106c922" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 19:36:22.114496 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.114461 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9"] Jul 20 19:36:22.118691 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.118661 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.121316 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.121292 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7jb2m\"" Jul 20 19:36:22.133773 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.133745 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9"] Jul 20 19:36:22.286847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.286798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36556998-8345-4788-80b0-21dd646c950f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.286860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.286903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36556998-8345-4788-80b0-21dd646c950f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.286937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.287015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.287051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36556998-8345-4788-80b0-21dd646c950f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.287087 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stsh6\" (UniqueName: \"kubernetes.io/projected/36556998-8345-4788-80b0-21dd646c950f-kube-api-access-stsh6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.287141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.287343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.287196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388267 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388267 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36556998-8345-4788-80b0-21dd646c950f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36556998-8345-4788-80b0-21dd646c950f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36556998-8345-4788-80b0-21dd646c950f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stsh6\" (UniqueName: \"kubernetes.io/projected/36556998-8345-4788-80b0-21dd646c950f-kube-api-access-stsh6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388777 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388777 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388608 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388876 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388855 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.388933 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.388881 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.389148 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.389118 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.389630 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.389311 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36556998-8345-4788-80b0-21dd646c950f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.392394 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.392332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36556998-8345-4788-80b0-21dd646c950f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.392741 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.392717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36556998-8345-4788-80b0-21dd646c950f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.400892 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.400866 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36556998-8345-4788-80b0-21dd646c950f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.401059 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.400998 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stsh6\" (UniqueName: \"kubernetes.io/projected/36556998-8345-4788-80b0-21dd646c950f-kube-api-access-stsh6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vnks9\" (UID: \"36556998-8345-4788-80b0-21dd646c950f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:22.436111 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:22.436077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:23.256029 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:23.255991 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9"] Jul 20 19:36:23.258097 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:36:23.258067 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36556998_8345_4788_80b0_21dd646c950f.slice/crio-c700611d10aa806d54d3e33a7ee0ad1ea8f76401a0664398d97d085135860bab WatchSource:0}: Error finding container c700611d10aa806d54d3e33a7ee0ad1ea8f76401a0664398d97d085135860bab: Status 404 returned error can't find the container with id c700611d10aa806d54d3e33a7ee0ad1ea8f76401a0664398d97d085135860bab Jul 20 19:36:23.884699 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:23.884663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" event={"ID":"375cbf6c-bbc2-46f6-8711-ea94b37ec6ff","Type":"ContainerStarted","Data":"278ad9ae6783b6399904a42253e52a03c51ff59942b7b08c700f357cb9b37784"} Jul 20 19:36:23.884887 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:23.884826 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:23.885805 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:23.885773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" event={"ID":"36556998-8345-4788-80b0-21dd646c950f","Type":"ContainerStarted","Data":"c700611d10aa806d54d3e33a7ee0ad1ea8f76401a0664398d97d085135860bab"} Jul 20 19:36:23.910160 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:23.910117 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" podStartSLOduration=1.3360681159999999 podStartE2EDuration="3.910103823s" totalImagesPullingTime="2.574035707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:23.90845049 +0000 UTC m=+525.533529485" watchObservedRunningTime="2026-07-20 19:36:23.910103823 +0000 UTC m=+525.535182817" Jul 20 19:36:24.833575 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:24.833542 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gxzbw" Jul 20 19:36:24.836037 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:24.836015 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hz8g8" Jul 20 19:36:24.874753 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:24.874715 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ppscm" Jul 20 19:36:24.895026 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:24.894986 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" event={"ID":"c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7","Type":"ContainerStarted","Data":"9ec07280f08b2c15985c1f78c453b1a09a98f793a5957baf8c9523b2e29572d3"} Jul 20 19:36:24.895786 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:24.895757 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:36:24.915644 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:24.915597 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" podStartSLOduration=1.826115507 podStartE2EDuration="4.915581542s" totalImagesPullingTime="3.089466035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:24.913944259 +0000 UTC m=+526.539023265" watchObservedRunningTime="2026-07-20 19:36:24.915581542 +0000 UTC m=+526.540660538" Jul 20 19:36:26.183522 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:26.183482 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:36:26.183922 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:26.183588 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:36:26.183922 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:26.183636 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:36:26.907175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:26.907137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" event={"ID":"36556998-8345-4788-80b0-21dd646c950f","Type":"ContainerStarted","Data":"f2606ccb77772f654201dee97aaf4f44bc647d80e751af3a57c8dcaa5522dfaa"} Jul 20 19:36:26.926919 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:26.926868 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" podStartSLOduration=2.003983915 podStartE2EDuration="4.926853204s" totalImagesPullingTime="2.922869289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:26.925520821 +0000 UTC m=+528.550599815" watchObservedRunningTime="2026-07-20 19:36:26.926853204 +0000 UTC m=+528.551932274" Jul 20 19:36:27.437009 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:27.436970 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:27.441508 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:27.441481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:27.912259 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:27.912226 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:27.913281 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:27.913263 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vnks9" Jul 20 19:36:54.899814 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:54.899729 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xjpq8" Jul 20 19:36:56.909368 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:36:56.909336 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gbdwj" Jul 20 19:37:10.902222 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:10.902182 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zrfs7"] Jul 20 19:37:10.905282 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:10.905254 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" Jul 20 19:37:10.909178 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:10.909149 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 19:37:10.909553 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:10.909530 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 19:37:10.909913 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:10.909892 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rr6lq\"" Jul 20 19:37:10.929555 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:10.929529 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zrfs7"] Jul 20 19:37:11.098855 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.098811 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvggr\" (UniqueName: \"kubernetes.io/projected/3deb8ebd-8b42-4b49-b2b3-173019e3abe7-kube-api-access-vvggr\") pod \"kuadrant-operator-catalog-zrfs7\" (UID: \"3deb8ebd-8b42-4b49-b2b3-173019e3abe7\") " pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" Jul 20 19:37:11.200074 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.199985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvggr\" (UniqueName: \"kubernetes.io/projected/3deb8ebd-8b42-4b49-b2b3-173019e3abe7-kube-api-access-vvggr\") pod \"kuadrant-operator-catalog-zrfs7\" (UID: \"3deb8ebd-8b42-4b49-b2b3-173019e3abe7\") " pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" Jul 20 19:37:11.208277 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.208247 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvggr\" (UniqueName: \"kubernetes.io/projected/3deb8ebd-8b42-4b49-b2b3-173019e3abe7-kube-api-access-vvggr\") pod \"kuadrant-operator-catalog-zrfs7\" (UID: \"3deb8ebd-8b42-4b49-b2b3-173019e3abe7\") " pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" Jul 20 19:37:11.215259 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.215233 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" Jul 20 19:37:11.240095 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.240065 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zrfs7"] Jul 20 19:37:11.344218 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.344196 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zrfs7"] Jul 20 19:37:11.345814 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:11.345786 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3deb8ebd_8b42_4b49_b2b3_173019e3abe7.slice/crio-55f65a1a2d6bada9e56e6770615e23bb3ff4bf332f88d03b30fcbc44dc861532 WatchSource:0}: Error finding container 55f65a1a2d6bada9e56e6770615e23bb3ff4bf332f88d03b30fcbc44dc861532: Status 404 returned error can't find the container with id 55f65a1a2d6bada9e56e6770615e23bb3ff4bf332f88d03b30fcbc44dc861532 Jul 20 19:37:11.452763 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.452685 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mzkh"] Jul 20 19:37:11.456853 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.456837 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:11.465985 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.465958 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mzkh"] Jul 20 19:37:11.604453 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.604399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq242\" (UniqueName: \"kubernetes.io/projected/722323c6-cb0a-4baa-920e-927667b7bc89-kube-api-access-zq242\") pod \"kuadrant-operator-catalog-2mzkh\" (UID: \"722323c6-cb0a-4baa-920e-927667b7bc89\") " pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:11.705175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.705093 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq242\" (UniqueName: \"kubernetes.io/projected/722323c6-cb0a-4baa-920e-927667b7bc89-kube-api-access-zq242\") pod \"kuadrant-operator-catalog-2mzkh\" (UID: \"722323c6-cb0a-4baa-920e-927667b7bc89\") " pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:11.714063 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.714035 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq242\" (UniqueName: \"kubernetes.io/projected/722323c6-cb0a-4baa-920e-927667b7bc89-kube-api-access-zq242\") pod \"kuadrant-operator-catalog-2mzkh\" (UID: \"722323c6-cb0a-4baa-920e-927667b7bc89\") " pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:11.766986 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.766949 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:11.913735 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:11.913713 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mzkh"] Jul 20 19:37:11.915342 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:11.915319 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod722323c6_cb0a_4baa_920e_927667b7bc89.slice/crio-54d4bcf014a4b6cb253ea14cab28e21a99907a41fccfcd3ea12874dbc7dff903 WatchSource:0}: Error finding container 54d4bcf014a4b6cb253ea14cab28e21a99907a41fccfcd3ea12874dbc7dff903: Status 404 returned error can't find the container with id 54d4bcf014a4b6cb253ea14cab28e21a99907a41fccfcd3ea12874dbc7dff903 Jul 20 19:37:12.112451 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:12.112407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" event={"ID":"3deb8ebd-8b42-4b49-b2b3-173019e3abe7","Type":"ContainerStarted","Data":"55f65a1a2d6bada9e56e6770615e23bb3ff4bf332f88d03b30fcbc44dc861532"} Jul 20 19:37:12.113572 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:12.113547 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" event={"ID":"722323c6-cb0a-4baa-920e-927667b7bc89","Type":"ContainerStarted","Data":"54d4bcf014a4b6cb253ea14cab28e21a99907a41fccfcd3ea12874dbc7dff903"} Jul 20 19:37:14.125704 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.125662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" event={"ID":"722323c6-cb0a-4baa-920e-927667b7bc89","Type":"ContainerStarted","Data":"bcbc63086d731b7770dafdebaa6260cd467ac4a4b91958585b9e18e12ff29c94"} Jul 20 19:37:14.127163 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.127137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" event={"ID":"3deb8ebd-8b42-4b49-b2b3-173019e3abe7","Type":"ContainerStarted","Data":"362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4"} Jul 20 19:37:14.127281 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.127204 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" podUID="3deb8ebd-8b42-4b49-b2b3-173019e3abe7" containerName="registry-server" containerID="cri-o://362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4" gracePeriod=2 Jul 20 19:37:14.145223 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.145169 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" podStartSLOduration=1.650450145 podStartE2EDuration="3.145152647s" totalImagesPullingTime="1.494702502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:37:14.144702157 +0000 UTC m=+575.769781152" watchObservedRunningTime="2026-07-20 19:37:14.145152647 +0000 UTC m=+575.770231654" Jul 20 19:37:14.365592 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.365567 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" Jul 20 19:37:14.428667 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.428590 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvggr\" (UniqueName: \"kubernetes.io/projected/3deb8ebd-8b42-4b49-b2b3-173019e3abe7-kube-api-access-vvggr\") pod \"3deb8ebd-8b42-4b49-b2b3-173019e3abe7\" (UID: \"3deb8ebd-8b42-4b49-b2b3-173019e3abe7\") " Jul 20 19:37:14.430750 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.430727 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3deb8ebd-8b42-4b49-b2b3-173019e3abe7-kube-api-access-vvggr" (OuterVolumeSpecName: "kube-api-access-vvggr") pod "3deb8ebd-8b42-4b49-b2b3-173019e3abe7" (UID: "3deb8ebd-8b42-4b49-b2b3-173019e3abe7"). InnerVolumeSpecName "kube-api-access-vvggr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:37:14.529995 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:14.529958 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvggr\" (UniqueName: \"kubernetes.io/projected/3deb8ebd-8b42-4b49-b2b3-173019e3abe7-kube-api-access-vvggr\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:15.133067 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.133032 2580 generic.go:356] "Generic (PLEG): container finished" podID="3deb8ebd-8b42-4b49-b2b3-173019e3abe7" containerID="362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4" exitCode=0 Jul 20 19:37:15.133495 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.133082 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" Jul 20 19:37:15.133495 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.133114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" event={"ID":"3deb8ebd-8b42-4b49-b2b3-173019e3abe7","Type":"ContainerDied","Data":"362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4"} Jul 20 19:37:15.133495 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.133183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zrfs7" event={"ID":"3deb8ebd-8b42-4b49-b2b3-173019e3abe7","Type":"ContainerDied","Data":"55f65a1a2d6bada9e56e6770615e23bb3ff4bf332f88d03b30fcbc44dc861532"} Jul 20 19:37:15.133495 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.133203 2580 scope.go:117] "RemoveContainer" containerID="362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4" Jul 20 19:37:15.141888 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.141857 2580 scope.go:117] "RemoveContainer" containerID="362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4" Jul 20 19:37:15.142135 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:37:15.142117 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4\": container with ID starting with 362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4 not found: ID does not exist" containerID="362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4" Jul 20 19:37:15.142177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.142144 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4"} err="failed to get container status \"362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4\": rpc error: code = NotFound desc = could not find container \"362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4\": container with ID starting with 362f0ee519ce32dd4fc0c62909b097de7fe37a4e3f35e6f78596a01a7a20e5a4 not found: ID does not exist" Jul 20 19:37:15.147574 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.147552 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zrfs7"] Jul 20 19:37:15.151378 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:15.151358 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zrfs7"] Jul 20 19:37:17.061582 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:17.061548 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3deb8ebd-8b42-4b49-b2b3-173019e3abe7" path="/var/lib/kubelet/pods/3deb8ebd-8b42-4b49-b2b3-173019e3abe7/volumes" Jul 20 19:37:21.767455 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:21.767394 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:21.767909 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:21.767474 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:21.791475 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:21.791446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:22.187864 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:22.187835 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2mzkh" Jul 20 19:37:26.492123 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.492086 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6"] Jul 20 19:37:26.492634 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.492447 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3deb8ebd-8b42-4b49-b2b3-173019e3abe7" containerName="registry-server" Jul 20 19:37:26.492634 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.492462 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3deb8ebd-8b42-4b49-b2b3-173019e3abe7" containerName="registry-server" Jul 20 19:37:26.492634 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.492517 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3deb8ebd-8b42-4b49-b2b3-173019e3abe7" containerName="registry-server" Jul 20 19:37:26.497359 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.497340 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.499697 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.499678 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5vdsb\"" Jul 20 19:37:26.507652 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.507632 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6"] Jul 20 19:37:26.528198 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.528168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2dz7\" (UniqueName: \"kubernetes.io/projected/d18c2e80-fd73-4678-8374-3ad5cd6538fd-kube-api-access-q2dz7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.528322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.528248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.528322 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.528277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.629051 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.629011 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2dz7\" (UniqueName: \"kubernetes.io/projected/d18c2e80-fd73-4678-8374-3ad5cd6538fd-kube-api-access-q2dz7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.629250 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.629073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.629250 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.629098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.629488 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.629463 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.629567 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.629509 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.637723 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.637697 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2dz7\" (UniqueName: \"kubernetes.io/projected/d18c2e80-fd73-4678-8374-3ad5cd6538fd-kube-api-access-q2dz7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.806937 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.806905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:26.895729 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.895696 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4"] Jul 20 19:37:26.900360 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.900337 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:26.914161 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.914058 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4"] Jul 20 19:37:26.932194 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.932161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:26.932347 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.932309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq482\" (UniqueName: \"kubernetes.io/projected/28a07d8c-af38-46d3-b1b3-1f778349da03-kube-api-access-xq482\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:26.932418 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.932350 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:26.944437 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:26.944398 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6"] Jul 20 19:37:26.946282 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:26.946252 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18c2e80_fd73_4678_8374_3ad5cd6538fd.slice/crio-e8886990b9ffd5d2b1c0e327c3b2cd23e65be426860ccd7a7d45973d07fa9125 WatchSource:0}: Error finding container e8886990b9ffd5d2b1c0e327c3b2cd23e65be426860ccd7a7d45973d07fa9125: Status 404 returned error can't find the container with id e8886990b9ffd5d2b1c0e327c3b2cd23e65be426860ccd7a7d45973d07fa9125 Jul 20 19:37:27.033705 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.033674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:27.033857 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.033757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq482\" (UniqueName: \"kubernetes.io/projected/28a07d8c-af38-46d3-b1b3-1f778349da03-kube-api-access-xq482\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:27.033857 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.033788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:27.034132 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.034111 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:27.034175 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.034121 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:27.041871 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.041845 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq482\" (UniqueName: \"kubernetes.io/projected/28a07d8c-af38-46d3-b1b3-1f778349da03-kube-api-access-xq482\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:27.188866 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.188775 2580 generic.go:356] "Generic (PLEG): container finished" podID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerID="f0a867fd1da2eca569db5370bc306c1e8bf73a821fcd3d2fc9cd077cf87f4bd7" exitCode=0 Jul 20 19:37:27.189008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.188859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" event={"ID":"d18c2e80-fd73-4678-8374-3ad5cd6538fd","Type":"ContainerDied","Data":"f0a867fd1da2eca569db5370bc306c1e8bf73a821fcd3d2fc9cd077cf87f4bd7"} Jul 20 19:37:27.189008 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.188915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" event={"ID":"d18c2e80-fd73-4678-8374-3ad5cd6538fd","Type":"ContainerStarted","Data":"e8886990b9ffd5d2b1c0e327c3b2cd23e65be426860ccd7a7d45973d07fa9125"} Jul 20 19:37:27.213356 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.213318 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:27.313919 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.313891 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v"] Jul 20 19:37:27.319162 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.319134 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.331572 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.331545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v"] Jul 20 19:37:27.356850 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.356800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4"] Jul 20 19:37:27.358473 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:27.358422 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a07d8c_af38_46d3_b1b3_1f778349da03.slice/crio-7bbdec309d474a3ee3a7c1c9dcf9afc1bb0cf09f7f6d02ce05e59c6b6c69c7da WatchSource:0}: Error finding container 7bbdec309d474a3ee3a7c1c9dcf9afc1bb0cf09f7f6d02ce05e59c6b6c69c7da: Status 404 returned error can't find the container with id 7bbdec309d474a3ee3a7c1c9dcf9afc1bb0cf09f7f6d02ce05e59c6b6c69c7da Jul 20 19:37:27.437095 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.437060 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb9fj\" (UniqueName: \"kubernetes.io/projected/0627a651-c27a-4990-bc5f-1b82ad647d95-kube-api-access-mb9fj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.437223 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.437140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.437223 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.437165 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.538010 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.537966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.538411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.538016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.538411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.538060 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb9fj\" (UniqueName: \"kubernetes.io/projected/0627a651-c27a-4990-bc5f-1b82ad647d95-kube-api-access-mb9fj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.538411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.538364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.538411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.538381 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.545846 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.545821 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb9fj\" (UniqueName: \"kubernetes.io/projected/0627a651-c27a-4990-bc5f-1b82ad647d95-kube-api-access-mb9fj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.629848 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.629811 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:27.704750 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.704716 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6"] Jul 20 19:37:27.709493 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.709406 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.718188 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.718162 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6"] Jul 20 19:37:27.740030 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.739998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.740185 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.740042 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.740251 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.740191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtdvz\" (UniqueName: \"kubernetes.io/projected/7143dbdb-4aaf-447d-8980-7554bb1249cc-kube-api-access-wtdvz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.765015 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.764986 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v"] Jul 20 19:37:27.766881 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:27.766854 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0627a651_c27a_4990_bc5f_1b82ad647d95.slice/crio-4395a9d8b07e8a93548cfa8c30ae052f9fa8bdc3e3a953070424d598ac1462da WatchSource:0}: Error finding container 4395a9d8b07e8a93548cfa8c30ae052f9fa8bdc3e3a953070424d598ac1462da: Status 404 returned error can't find the container with id 4395a9d8b07e8a93548cfa8c30ae052f9fa8bdc3e3a953070424d598ac1462da Jul 20 19:37:27.841722 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.841689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtdvz\" (UniqueName: \"kubernetes.io/projected/7143dbdb-4aaf-447d-8980-7554bb1249cc-kube-api-access-wtdvz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.841883 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.841764 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.841883 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.841788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.842146 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.842117 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.842216 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.842152 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:27.852350 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:27.852322 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtdvz\" (UniqueName: \"kubernetes.io/projected/7143dbdb-4aaf-447d-8980-7554bb1249cc-kube-api-access-wtdvz\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:28.023952 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.023926 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:28.149269 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.149242 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6"] Jul 20 19:37:28.150772 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:28.150749 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7143dbdb_4aaf_447d_8980_7554bb1249cc.slice/crio-ec72213ff5d415c30ad5a03da136ab806893678ec88b649fe5cbc8f7e111ec99 WatchSource:0}: Error finding container ec72213ff5d415c30ad5a03da136ab806893678ec88b649fe5cbc8f7e111ec99: Status 404 returned error can't find the container with id ec72213ff5d415c30ad5a03da136ab806893678ec88b649fe5cbc8f7e111ec99 Jul 20 19:37:28.195575 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.195547 2580 generic.go:356] "Generic (PLEG): container finished" podID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerID="52b6c2bde775a3dbfb91103d43424f8e94f8a5e34e18a13cfd6f3a7d7abe920f" exitCode=0 Jul 20 19:37:28.195679 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.195629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" event={"ID":"d18c2e80-fd73-4678-8374-3ad5cd6538fd","Type":"ContainerDied","Data":"52b6c2bde775a3dbfb91103d43424f8e94f8a5e34e18a13cfd6f3a7d7abe920f"} Jul 20 19:37:28.196940 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.196915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" event={"ID":"7143dbdb-4aaf-447d-8980-7554bb1249cc","Type":"ContainerStarted","Data":"ec72213ff5d415c30ad5a03da136ab806893678ec88b649fe5cbc8f7e111ec99"} Jul 20 19:37:28.198269 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.198251 2580 generic.go:356] "Generic (PLEG): container finished" podID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerID="c19308c7e6c216b20ce312468a42feaabd76905ac19f9a9ae4ecd3890d490cbc" exitCode=0 Jul 20 19:37:28.198358 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.198334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" event={"ID":"28a07d8c-af38-46d3-b1b3-1f778349da03","Type":"ContainerDied","Data":"c19308c7e6c216b20ce312468a42feaabd76905ac19f9a9ae4ecd3890d490cbc"} Jul 20 19:37:28.198450 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.198375 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" event={"ID":"28a07d8c-af38-46d3-b1b3-1f778349da03","Type":"ContainerStarted","Data":"7bbdec309d474a3ee3a7c1c9dcf9afc1bb0cf09f7f6d02ce05e59c6b6c69c7da"} Jul 20 19:37:28.199829 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.199808 2580 generic.go:356] "Generic (PLEG): container finished" podID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerID="e6b57fe579efbf35789846352d88e3fd11aa9f7cd67a5cef1703df144884ecb1" exitCode=0 Jul 20 19:37:28.199925 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.199881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" event={"ID":"0627a651-c27a-4990-bc5f-1b82ad647d95","Type":"ContainerDied","Data":"e6b57fe579efbf35789846352d88e3fd11aa9f7cd67a5cef1703df144884ecb1"} Jul 20 19:37:28.199925 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:28.199914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" event={"ID":"0627a651-c27a-4990-bc5f-1b82ad647d95","Type":"ContainerStarted","Data":"4395a9d8b07e8a93548cfa8c30ae052f9fa8bdc3e3a953070424d598ac1462da"} Jul 20 19:37:29.212365 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:29.212333 2580 generic.go:356] "Generic (PLEG): container finished" podID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerID="fbddc7953b65ae79872d00c0189e648e0db19c4108aa6cb47654cccbd5094e67" exitCode=0 Jul 20 19:37:29.212783 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:29.212541 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" event={"ID":"d18c2e80-fd73-4678-8374-3ad5cd6538fd","Type":"ContainerDied","Data":"fbddc7953b65ae79872d00c0189e648e0db19c4108aa6cb47654cccbd5094e67"} Jul 20 19:37:29.217315 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:29.217252 2580 generic.go:356] "Generic (PLEG): container finished" podID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerID="e887ca458704d69874c8cf89b498cbf30868a0f418b0b7f877d3814743829311" exitCode=0 Jul 20 19:37:29.217421 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:29.217397 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" event={"ID":"7143dbdb-4aaf-447d-8980-7554bb1249cc","Type":"ContainerDied","Data":"e887ca458704d69874c8cf89b498cbf30868a0f418b0b7f877d3814743829311"} Jul 20 19:37:30.229847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.229751 2580 generic.go:356] "Generic (PLEG): container finished" podID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerID="1681621fc388ab0bf527b6e749a5bb6c14d4c6f49f44c1d2c36500f352f39b6b" exitCode=0 Jul 20 19:37:30.229847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.229836 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" event={"ID":"7143dbdb-4aaf-447d-8980-7554bb1249cc","Type":"ContainerDied","Data":"1681621fc388ab0bf527b6e749a5bb6c14d4c6f49f44c1d2c36500f352f39b6b"} Jul 20 19:37:30.231877 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.231851 2580 generic.go:356] "Generic (PLEG): container finished" podID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerID="ad88f3acb04f0c18267fae225a44cb64b4d64db454d8df9fe49a983bb2806b7a" exitCode=0 Jul 20 19:37:30.231988 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.231936 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" event={"ID":"28a07d8c-af38-46d3-b1b3-1f778349da03","Type":"ContainerDied","Data":"ad88f3acb04f0c18267fae225a44cb64b4d64db454d8df9fe49a983bb2806b7a"} Jul 20 19:37:30.234023 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.234004 2580 generic.go:356] "Generic (PLEG): container finished" podID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerID="19eb4b13c809dbca299196d962366dc730931b9b6b7f6b9d1ba5d44330442a90" exitCode=0 Jul 20 19:37:30.234114 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.234078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" event={"ID":"0627a651-c27a-4990-bc5f-1b82ad647d95","Type":"ContainerDied","Data":"19eb4b13c809dbca299196d962366dc730931b9b6b7f6b9d1ba5d44330442a90"} Jul 20 19:37:30.408888 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.408859 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:30.467727 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.467702 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-util\") pod \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " Jul 20 19:37:30.467828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.467806 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2dz7\" (UniqueName: \"kubernetes.io/projected/d18c2e80-fd73-4678-8374-3ad5cd6538fd-kube-api-access-q2dz7\") pod \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " Jul 20 19:37:30.467865 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.467836 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-bundle\") pod \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\" (UID: \"d18c2e80-fd73-4678-8374-3ad5cd6538fd\") " Jul 20 19:37:30.468370 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.468346 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-bundle" (OuterVolumeSpecName: "bundle") pod "d18c2e80-fd73-4678-8374-3ad5cd6538fd" (UID: "d18c2e80-fd73-4678-8374-3ad5cd6538fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:30.469828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.469804 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18c2e80-fd73-4678-8374-3ad5cd6538fd-kube-api-access-q2dz7" (OuterVolumeSpecName: "kube-api-access-q2dz7") pod "d18c2e80-fd73-4678-8374-3ad5cd6538fd" (UID: "d18c2e80-fd73-4678-8374-3ad5cd6538fd"). InnerVolumeSpecName "kube-api-access-q2dz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:37:30.473535 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.473513 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-util" (OuterVolumeSpecName: "util") pod "d18c2e80-fd73-4678-8374-3ad5cd6538fd" (UID: "d18c2e80-fd73-4678-8374-3ad5cd6538fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:30.569295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.569269 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:30.569295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.569295 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18c2e80-fd73-4678-8374-3ad5cd6538fd-util\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:30.569422 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:30.569304 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2dz7\" (UniqueName: \"kubernetes.io/projected/d18c2e80-fd73-4678-8374-3ad5cd6538fd-kube-api-access-q2dz7\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:31.240913 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.240826 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" Jul 20 19:37:31.240913 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.240819 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6" event={"ID":"d18c2e80-fd73-4678-8374-3ad5cd6538fd","Type":"ContainerDied","Data":"e8886990b9ffd5d2b1c0e327c3b2cd23e65be426860ccd7a7d45973d07fa9125"} Jul 20 19:37:31.241440 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.240955 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8886990b9ffd5d2b1c0e327c3b2cd23e65be426860ccd7a7d45973d07fa9125" Jul 20 19:37:31.243344 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.243321 2580 generic.go:356] "Generic (PLEG): container finished" podID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerID="8d027ef2aeae566407590373a0a59669861941b8e8ceef6200081de5b594bc3f" exitCode=0 Jul 20 19:37:31.243457 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.243361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" event={"ID":"7143dbdb-4aaf-447d-8980-7554bb1249cc","Type":"ContainerDied","Data":"8d027ef2aeae566407590373a0a59669861941b8e8ceef6200081de5b594bc3f"} Jul 20 19:37:31.245281 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.245262 2580 generic.go:356] "Generic (PLEG): container finished" podID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerID="514f1fad1f13ecf7c4e43c8b9a41df29d687c33cf82346292b61c8d99fab3cfd" exitCode=0 Jul 20 19:37:31.245384 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.245337 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" event={"ID":"28a07d8c-af38-46d3-b1b3-1f778349da03","Type":"ContainerDied","Data":"514f1fad1f13ecf7c4e43c8b9a41df29d687c33cf82346292b61c8d99fab3cfd"} Jul 20 19:37:31.247474 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.247455 2580 generic.go:356] "Generic (PLEG): container finished" podID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerID="75e2b3f7ae43b1774b08c73104987142efbada1ecb0a5cd9ee94c2de81594210" exitCode=0 Jul 20 19:37:31.247554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:31.247527 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" event={"ID":"0627a651-c27a-4990-bc5f-1b82ad647d95","Type":"ContainerDied","Data":"75e2b3f7ae43b1774b08c73104987142efbada1ecb0a5cd9ee94c2de81594210"} Jul 20 19:37:32.411150 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.411122 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:32.446067 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.446041 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:32.448820 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.448802 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:32.488136 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.488107 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb9fj\" (UniqueName: \"kubernetes.io/projected/0627a651-c27a-4990-bc5f-1b82ad647d95-kube-api-access-mb9fj\") pod \"0627a651-c27a-4990-bc5f-1b82ad647d95\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " Jul 20 19:37:32.488295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.488146 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-util\") pod \"28a07d8c-af38-46d3-b1b3-1f778349da03\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " Jul 20 19:37:32.488295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.488168 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-util\") pod \"0627a651-c27a-4990-bc5f-1b82ad647d95\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " Jul 20 19:37:32.488295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.488216 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq482\" (UniqueName: \"kubernetes.io/projected/28a07d8c-af38-46d3-b1b3-1f778349da03-kube-api-access-xq482\") pod \"28a07d8c-af38-46d3-b1b3-1f778349da03\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " Jul 20 19:37:32.488295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.488272 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-bundle\") pod \"28a07d8c-af38-46d3-b1b3-1f778349da03\" (UID: \"28a07d8c-af38-46d3-b1b3-1f778349da03\") " Jul 20 19:37:32.488536 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.488312 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-bundle\") pod \"0627a651-c27a-4990-bc5f-1b82ad647d95\" (UID: \"0627a651-c27a-4990-bc5f-1b82ad647d95\") " Jul 20 19:37:32.489187 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.489153 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-bundle" (OuterVolumeSpecName: "bundle") pod "0627a651-c27a-4990-bc5f-1b82ad647d95" (UID: "0627a651-c27a-4990-bc5f-1b82ad647d95"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:32.489728 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.489677 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-bundle" (OuterVolumeSpecName: "bundle") pod "28a07d8c-af38-46d3-b1b3-1f778349da03" (UID: "28a07d8c-af38-46d3-b1b3-1f778349da03"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:32.491060 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.491023 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a07d8c-af38-46d3-b1b3-1f778349da03-kube-api-access-xq482" (OuterVolumeSpecName: "kube-api-access-xq482") pod "28a07d8c-af38-46d3-b1b3-1f778349da03" (UID: "28a07d8c-af38-46d3-b1b3-1f778349da03"). InnerVolumeSpecName "kube-api-access-xq482". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:37:32.491860 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.491823 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0627a651-c27a-4990-bc5f-1b82ad647d95-kube-api-access-mb9fj" (OuterVolumeSpecName: "kube-api-access-mb9fj") pod "0627a651-c27a-4990-bc5f-1b82ad647d95" (UID: "0627a651-c27a-4990-bc5f-1b82ad647d95"). InnerVolumeSpecName "kube-api-access-mb9fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:37:32.497328 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.497302 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-util" (OuterVolumeSpecName: "util") pod "28a07d8c-af38-46d3-b1b3-1f778349da03" (UID: "28a07d8c-af38-46d3-b1b3-1f778349da03"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:32.497674 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.497650 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-util" (OuterVolumeSpecName: "util") pod "0627a651-c27a-4990-bc5f-1b82ad647d95" (UID: "0627a651-c27a-4990-bc5f-1b82ad647d95"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:32.589884 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.589857 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-bundle\") pod \"7143dbdb-4aaf-447d-8980-7554bb1249cc\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " Jul 20 19:37:32.590028 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.589915 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtdvz\" (UniqueName: \"kubernetes.io/projected/7143dbdb-4aaf-447d-8980-7554bb1249cc-kube-api-access-wtdvz\") pod \"7143dbdb-4aaf-447d-8980-7554bb1249cc\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " Jul 20 19:37:32.590028 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.589980 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-util\") pod \"7143dbdb-4aaf-447d-8980-7554bb1249cc\" (UID: \"7143dbdb-4aaf-447d-8980-7554bb1249cc\") " Jul 20 19:37:32.590153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.590099 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.590153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.590109 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.590153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.590118 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb9fj\" (UniqueName: \"kubernetes.io/projected/0627a651-c27a-4990-bc5f-1b82ad647d95-kube-api-access-mb9fj\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.590153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.590126 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28a07d8c-af38-46d3-b1b3-1f778349da03-util\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.590153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.590134 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0627a651-c27a-4990-bc5f-1b82ad647d95-util\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.590153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.590141 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq482\" (UniqueName: \"kubernetes.io/projected/28a07d8c-af38-46d3-b1b3-1f778349da03-kube-api-access-xq482\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.590504 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.590455 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-bundle" (OuterVolumeSpecName: "bundle") pod "7143dbdb-4aaf-447d-8980-7554bb1249cc" (UID: "7143dbdb-4aaf-447d-8980-7554bb1249cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:32.591910 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.591878 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7143dbdb-4aaf-447d-8980-7554bb1249cc-kube-api-access-wtdvz" (OuterVolumeSpecName: "kube-api-access-wtdvz") pod "7143dbdb-4aaf-447d-8980-7554bb1249cc" (UID: "7143dbdb-4aaf-447d-8980-7554bb1249cc"). InnerVolumeSpecName "kube-api-access-wtdvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:37:32.595115 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.595094 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-util" (OuterVolumeSpecName: "util") pod "7143dbdb-4aaf-447d-8980-7554bb1249cc" (UID: "7143dbdb-4aaf-447d-8980-7554bb1249cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:37:32.691169 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.691133 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-util\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.691169 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.691167 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7143dbdb-4aaf-447d-8980-7554bb1249cc-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:32.691349 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:32.691183 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtdvz\" (UniqueName: \"kubernetes.io/projected/7143dbdb-4aaf-447d-8980-7554bb1249cc-kube-api-access-wtdvz\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:37:33.258061 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.258031 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" Jul 20 19:37:33.258061 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.258055 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v" event={"ID":"0627a651-c27a-4990-bc5f-1b82ad647d95","Type":"ContainerDied","Data":"4395a9d8b07e8a93548cfa8c30ae052f9fa8bdc3e3a953070424d598ac1462da"} Jul 20 19:37:33.258297 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.258101 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4395a9d8b07e8a93548cfa8c30ae052f9fa8bdc3e3a953070424d598ac1462da" Jul 20 19:37:33.259951 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.259928 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" Jul 20 19:37:33.259951 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.259935 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6" event={"ID":"7143dbdb-4aaf-447d-8980-7554bb1249cc","Type":"ContainerDied","Data":"ec72213ff5d415c30ad5a03da136ab806893678ec88b649fe5cbc8f7e111ec99"} Jul 20 19:37:33.260164 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.259977 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec72213ff5d415c30ad5a03da136ab806893678ec88b649fe5cbc8f7e111ec99" Jul 20 19:37:33.262160 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.262137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" event={"ID":"28a07d8c-af38-46d3-b1b3-1f778349da03","Type":"ContainerDied","Data":"7bbdec309d474a3ee3a7c1c9dcf9afc1bb0cf09f7f6d02ce05e59c6b6c69c7da"} Jul 20 19:37:33.262267 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.262177 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bbdec309d474a3ee3a7c1c9dcf9afc1bb0cf09f7f6d02ce05e59c6b6c69c7da" Jul 20 19:37:33.262267 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:33.262184 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4" Jul 20 19:37:42.440354 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440318 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw"] Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440667 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440679 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440691 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440697 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440709 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440714 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440721 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440726 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440735 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440740 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440746 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440751 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440757 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440762 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerName="util" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440769 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440773 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440779 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440784 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440791 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440797 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440803 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440808 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerName="pull" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440815 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440819 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440867 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7143dbdb-4aaf-447d-8980-7554bb1249cc" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440876 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d18c2e80-fd73-4678-8374-3ad5cd6538fd" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440884 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a07d8c-af38-46d3-b1b3-1f778349da03" containerName="extract" Jul 20 19:37:42.442670 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.440890 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0627a651-c27a-4990-bc5f-1b82ad647d95" containerName="extract" Jul 20 19:37:42.443632 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.443615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:37:42.446007 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.445984 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-prdzp\"" Jul 20 19:37:42.454304 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.454284 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw"] Jul 20 19:37:42.582066 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.582027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsn5p\" (UniqueName: \"kubernetes.io/projected/f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067-kube-api-access-zsn5p\") pod \"limitador-operator-controller-manager-859d7b78bd-x69vw\" (UID: \"f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:37:42.683557 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.683515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsn5p\" (UniqueName: \"kubernetes.io/projected/f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067-kube-api-access-zsn5p\") pod \"limitador-operator-controller-manager-859d7b78bd-x69vw\" (UID: \"f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:37:42.696205 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.696136 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsn5p\" (UniqueName: \"kubernetes.io/projected/f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067-kube-api-access-zsn5p\") pod \"limitador-operator-controller-manager-859d7b78bd-x69vw\" (UID: \"f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:37:42.753238 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.753205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:37:42.881455 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:42.881422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw"] Jul 20 19:37:42.883794 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:42.883762 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d4e400_fb96_4e1b_9a2d_40cf4fdbf067.slice/crio-745354772ff32226d30bf33690cadfe8ca4b52a6f0b96961a3f24e8d1ac41ab6 WatchSource:0}: Error finding container 745354772ff32226d30bf33690cadfe8ca4b52a6f0b96961a3f24e8d1ac41ab6: Status 404 returned error can't find the container with id 745354772ff32226d30bf33690cadfe8ca4b52a6f0b96961a3f24e8d1ac41ab6 Jul 20 19:37:43.315631 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:43.315588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" event={"ID":"f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067","Type":"ContainerStarted","Data":"745354772ff32226d30bf33690cadfe8ca4b52a6f0b96961a3f24e8d1ac41ab6"} Jul 20 19:37:45.328942 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:45.328899 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" event={"ID":"f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067","Type":"ContainerStarted","Data":"5e182c9b87cb914ead20e804fb9bcf8e8a55b82a016663ad0cc96a4ef8b5f153"} Jul 20 19:37:45.329400 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:45.328989 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:37:45.349498 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:45.349448 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" podStartSLOduration=1.6667865370000001 podStartE2EDuration="3.349417636s" totalImagesPullingTime="1.682631099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:37:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:37:45.346870001 +0000 UTC m=+606.971948996" watchObservedRunningTime="2026-07-20 19:37:45.349417636 +0000 UTC m=+606.974496630" Jul 20 19:37:47.604959 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.604922 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls"] Jul 20 19:37:47.613239 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.613213 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:47.615477 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.615457 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7g7tv\"" Jul 20 19:37:47.627643 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.627621 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls"] Jul 20 19:37:47.730282 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.730249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk2gx\" (UniqueName: \"kubernetes.io/projected/77cbc415-1d85-4497-894e-1d2ed1319b2a-kube-api-access-tk2gx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:47.730470 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.730311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77cbc415-1d85-4497-894e-1d2ed1319b2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:47.831375 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.831326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk2gx\" (UniqueName: \"kubernetes.io/projected/77cbc415-1d85-4497-894e-1d2ed1319b2a-kube-api-access-tk2gx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:47.831570 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.831454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77cbc415-1d85-4497-894e-1d2ed1319b2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:47.831798 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.831777 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77cbc415-1d85-4497-894e-1d2ed1319b2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:47.845808 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.845779 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk2gx\" (UniqueName: \"kubernetes.io/projected/77cbc415-1d85-4497-894e-1d2ed1319b2a-kube-api-access-tk2gx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:47.922822 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:47.922737 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:48.063259 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:48.063232 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls"] Jul 20 19:37:48.066608 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:48.066581 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77cbc415_1d85_4497_894e_1d2ed1319b2a.slice/crio-96daf5ece37089222d927e3a86b265d2ed328b7382b5dd9a9ab5a0d15902c8cd WatchSource:0}: Error finding container 96daf5ece37089222d927e3a86b265d2ed328b7382b5dd9a9ab5a0d15902c8cd: Status 404 returned error can't find the container with id 96daf5ece37089222d927e3a86b265d2ed328b7382b5dd9a9ab5a0d15902c8cd Jul 20 19:37:48.343911 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:48.343868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" event={"ID":"77cbc415-1d85-4497-894e-1d2ed1319b2a","Type":"ContainerStarted","Data":"96daf5ece37089222d927e3a86b265d2ed328b7382b5dd9a9ab5a0d15902c8cd"} Jul 20 19:37:50.543343 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.543301 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g"] Jul 20 19:37:50.546878 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.546860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" Jul 20 19:37:50.548947 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.548928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kwxpk\"" Jul 20 19:37:50.556093 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.556073 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g"] Jul 20 19:37:50.657439 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.657387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54zl5\" (UniqueName: \"kubernetes.io/projected/9d71df69-24d2-49b0-bba9-e5ff4238e93b-kube-api-access-54zl5\") pod \"authorino-operator-75cc8d5cb7-t8k6g\" (UID: \"9d71df69-24d2-49b0-bba9-e5ff4238e93b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" Jul 20 19:37:50.758108 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.758068 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54zl5\" (UniqueName: \"kubernetes.io/projected/9d71df69-24d2-49b0-bba9-e5ff4238e93b-kube-api-access-54zl5\") pod \"authorino-operator-75cc8d5cb7-t8k6g\" (UID: \"9d71df69-24d2-49b0-bba9-e5ff4238e93b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" Jul 20 19:37:50.782959 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.782930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54zl5\" (UniqueName: \"kubernetes.io/projected/9d71df69-24d2-49b0-bba9-e5ff4238e93b-kube-api-access-54zl5\") pod \"authorino-operator-75cc8d5cb7-t8k6g\" (UID: \"9d71df69-24d2-49b0-bba9-e5ff4238e93b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" Jul 20 19:37:50.857579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:50.857499 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" Jul 20 19:37:51.003890 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:51.003865 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g"] Jul 20 19:37:51.005406 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:37:51.005379 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d71df69_24d2_49b0_bba9_e5ff4238e93b.slice/crio-d2b698a21170fd1279465a834a2b32c3239c772ed15ebf869b2741b7f458d98d WatchSource:0}: Error finding container d2b698a21170fd1279465a834a2b32c3239c772ed15ebf869b2741b7f458d98d: Status 404 returned error can't find the container with id d2b698a21170fd1279465a834a2b32c3239c772ed15ebf869b2741b7f458d98d Jul 20 19:37:51.364026 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:51.363978 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" event={"ID":"9d71df69-24d2-49b0-bba9-e5ff4238e93b","Type":"ContainerStarted","Data":"d2b698a21170fd1279465a834a2b32c3239c772ed15ebf869b2741b7f458d98d"} Jul 20 19:37:55.388899 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:55.388856 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" event={"ID":"77cbc415-1d85-4497-894e-1d2ed1319b2a","Type":"ContainerStarted","Data":"d3e2adfd42ae85a250f0da9a7505649a0fe20d837f21b90f316432af4d53d7d8"} Jul 20 19:37:55.389324 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:55.388929 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:37:55.390598 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:55.390565 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" event={"ID":"9d71df69-24d2-49b0-bba9-e5ff4238e93b","Type":"ContainerStarted","Data":"89365131633e1bd40046faa3229e6669c84b76828d42ee02baa56e73d600f483"} Jul 20 19:37:55.390742 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:55.390642 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" Jul 20 19:37:55.414794 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:55.414744 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" podStartSLOduration=2.161849511 podStartE2EDuration="8.414729563s" totalImagesPullingTime="6.252880052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:37:55.412788032 +0000 UTC m=+617.037867060" watchObservedRunningTime="2026-07-20 19:37:55.414729563 +0000 UTC m=+617.039808558" Jul 20 19:37:55.431688 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:55.431643 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" podStartSLOduration=2.111977322 podStartE2EDuration="5.431628615s" totalImagesPullingTime="3.319651293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:37:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:37:55.430633911 +0000 UTC m=+617.055712921" watchObservedRunningTime="2026-07-20 19:37:55.431628615 +0000 UTC m=+617.056707609" Jul 20 19:37:56.335953 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:37:56.335922 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:38:06.399767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:06.399686 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:06.400224 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:06.399923 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t8k6g" Jul 20 19:38:08.285485 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.285446 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls"] Jul 20 19:38:08.285875 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.285675 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" containerName="manager" containerID="cri-o://d3e2adfd42ae85a250f0da9a7505649a0fe20d837f21b90f316432af4d53d7d8" gracePeriod=2 Jul 20 19:38:08.299554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.299520 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls"] Jul 20 19:38:08.300995 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.300959 2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"77cbc415-1d85-4497-894e-1d2ed1319b2a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-lnrls\": pods \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:08.315482 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.315446 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd"] Jul 20 19:38:08.315914 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.315883 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" containerName="manager" Jul 20 19:38:08.316013 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.315921 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" containerName="manager" Jul 20 19:38:08.316099 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.316041 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" containerName="manager" Jul 20 19:38:08.319292 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.319272 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.319695 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.319658 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw"] Jul 20 19:38:08.319948 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.319881 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" containerName="manager" containerID="cri-o://5e182c9b87cb914ead20e804fb9bcf8e8a55b82a016663ad0cc96a4ef8b5f153" gracePeriod=2 Jul 20 19:38:08.323276 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.323255 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw"] Jul 20 19:38:08.338392 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.338370 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd"] Jul 20 19:38:08.348416 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.348394 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr"] Jul 20 19:38:08.348734 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.348721 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" containerName="manager" Jul 20 19:38:08.348787 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.348736 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" containerName="manager" Jul 20 19:38:08.348823 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.348799 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" containerName="manager" Jul 20 19:38:08.351792 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.351773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" Jul 20 19:38:08.361135 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.361085 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:08.362624 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.362591 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x69vw\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:38:08.369772 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.369746 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr"] Jul 20 19:38:08.389190 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.389161 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:08.413066 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.413028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxttx\" (UniqueName: \"kubernetes.io/projected/21676113-5e47-4076-9588-9782f1a5b2d2-kube-api-access-mxttx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5kcqd\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.413169 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.413106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9b4c\" (UniqueName: \"kubernetes.io/projected/6954ccee-eaea-402a-8894-49e29808da89-kube-api-access-c9b4c\") pod \"limitador-operator-controller-manager-859d7b78bd-7gkqr\" (UID: \"6954ccee-eaea-402a-8894-49e29808da89\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" Jul 20 19:38:08.413169 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.413127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21676113-5e47-4076-9588-9782f1a5b2d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5kcqd\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.464321 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.464273 2580 generic.go:356] "Generic (PLEG): container finished" podID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" containerID="5e182c9b87cb914ead20e804fb9bcf8e8a55b82a016663ad0cc96a4ef8b5f153" exitCode=0 Jul 20 19:38:08.466018 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.465990 2580 generic.go:356] "Generic (PLEG): container finished" podID="77cbc415-1d85-4497-894e-1d2ed1319b2a" containerID="d3e2adfd42ae85a250f0da9a7505649a0fe20d837f21b90f316432af4d53d7d8" exitCode=0 Jul 20 19:38:08.514022 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.513988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9b4c\" (UniqueName: \"kubernetes.io/projected/6954ccee-eaea-402a-8894-49e29808da89-kube-api-access-c9b4c\") pod \"limitador-operator-controller-manager-859d7b78bd-7gkqr\" (UID: \"6954ccee-eaea-402a-8894-49e29808da89\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" Jul 20 19:38:08.514159 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.514049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21676113-5e47-4076-9588-9782f1a5b2d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5kcqd\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.514159 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.514135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxttx\" (UniqueName: \"kubernetes.io/projected/21676113-5e47-4076-9588-9782f1a5b2d2-kube-api-access-mxttx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5kcqd\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.514553 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.514516 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21676113-5e47-4076-9588-9782f1a5b2d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5kcqd\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.550177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.550101 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxttx\" (UniqueName: \"kubernetes.io/projected/21676113-5e47-4076-9588-9782f1a5b2d2-kube-api-access-mxttx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5kcqd\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.550177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.550140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9b4c\" (UniqueName: \"kubernetes.io/projected/6954ccee-eaea-402a-8894-49e29808da89-kube-api-access-c9b4c\") pod \"limitador-operator-controller-manager-859d7b78bd-7gkqr\" (UID: \"6954ccee-eaea-402a-8894-49e29808da89\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" Jul 20 19:38:08.583700 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.583676 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:08.585727 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.585697 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x69vw\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:38:08.585828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.585712 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:38:08.591996 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.591969 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:08.594052 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.594027 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:08.595700 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:08.595680 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x69vw\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:38:08.614709 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.614688 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77cbc415-1d85-4497-894e-1d2ed1319b2a-extensions-socket-volume\") pod \"77cbc415-1d85-4497-894e-1d2ed1319b2a\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " Jul 20 19:38:08.614794 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.614756 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsn5p\" (UniqueName: \"kubernetes.io/projected/f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067-kube-api-access-zsn5p\") pod \"f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067\" (UID: \"f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067\") " Jul 20 19:38:08.614794 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.614779 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk2gx\" (UniqueName: \"kubernetes.io/projected/77cbc415-1d85-4497-894e-1d2ed1319b2a-kube-api-access-tk2gx\") pod \"77cbc415-1d85-4497-894e-1d2ed1319b2a\" (UID: \"77cbc415-1d85-4497-894e-1d2ed1319b2a\") " Jul 20 19:38:08.615121 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.615096 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77cbc415-1d85-4497-894e-1d2ed1319b2a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "77cbc415-1d85-4497-894e-1d2ed1319b2a" (UID: "77cbc415-1d85-4497-894e-1d2ed1319b2a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:38:08.616831 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.616803 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067-kube-api-access-zsn5p" (OuterVolumeSpecName: "kube-api-access-zsn5p") pod "f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" (UID: "f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067"). InnerVolumeSpecName "kube-api-access-zsn5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:38:08.616898 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.616838 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77cbc415-1d85-4497-894e-1d2ed1319b2a-kube-api-access-tk2gx" (OuterVolumeSpecName: "kube-api-access-tk2gx") pod "77cbc415-1d85-4497-894e-1d2ed1319b2a" (UID: "77cbc415-1d85-4497-894e-1d2ed1319b2a"). InnerVolumeSpecName "kube-api-access-tk2gx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:38:08.716274 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.716244 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsn5p\" (UniqueName: \"kubernetes.io/projected/f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067-kube-api-access-zsn5p\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:38:08.716274 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.716273 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk2gx\" (UniqueName: \"kubernetes.io/projected/77cbc415-1d85-4497-894e-1d2ed1319b2a-kube-api-access-tk2gx\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:38:08.716474 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.716282 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77cbc415-1d85-4497-894e-1d2ed1319b2a-extensions-socket-volume\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:38:08.729118 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.729083 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:08.735039 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.735013 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" Jul 20 19:38:08.875769 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.875744 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd"] Jul 20 19:38:08.878626 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:38:08.878598 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21676113_5e47_4076_9588_9782f1a5b2d2.slice/crio-ea3e003a638978920f8f8b189ec0a8f0b0265952b1ede8ceb19406e383bfbe3c WatchSource:0}: Error finding container ea3e003a638978920f8f8b189ec0a8f0b0265952b1ede8ceb19406e383bfbe3c: Status 404 returned error can't find the container with id ea3e003a638978920f8f8b189ec0a8f0b0265952b1ede8ceb19406e383bfbe3c Jul 20 19:38:08.895463 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:08.895440 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr"] Jul 20 19:38:08.897452 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:38:08.897408 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6954ccee_eaea_402a_8894_49e29808da89.slice/crio-ccbafd0c4fd4227100fe381d667b8a89a3a14bdb5dd6d19937e9db2da32dd31c WatchSource:0}: Error finding container ccbafd0c4fd4227100fe381d667b8a89a3a14bdb5dd6d19937e9db2da32dd31c: Status 404 returned error can't find the container with id ccbafd0c4fd4227100fe381d667b8a89a3a14bdb5dd6d19937e9db2da32dd31c Jul 20 19:38:09.059392 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:09.059357 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x69vw\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:38:09.060704 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:09.060673 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lnrls\" is forbidden: User \"system:node:ip-10-0-132-219.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-219.ec2.internal' and this object" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:09.061162 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.061136 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77cbc415-1d85-4497-894e-1d2ed1319b2a" path="/var/lib/kubelet/pods/77cbc415-1d85-4497-894e-1d2ed1319b2a/volumes" Jul 20 19:38:09.061616 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.061597 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067" path="/var/lib/kubelet/pods/f4d4e400-fb96-4e1b-9a2d-40cf4fdbf067/volumes" Jul 20 19:38:09.473104 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.473024 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x69vw" Jul 20 19:38:09.473104 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.473032 2580 scope.go:117] "RemoveContainer" containerID="5e182c9b87cb914ead20e804fb9bcf8e8a55b82a016663ad0cc96a4ef8b5f153" Jul 20 19:38:09.474858 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.474834 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lnrls" Jul 20 19:38:09.476961 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.476926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" event={"ID":"21676113-5e47-4076-9588-9782f1a5b2d2","Type":"ContainerStarted","Data":"2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6"} Jul 20 19:38:09.477069 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.477019 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" event={"ID":"21676113-5e47-4076-9588-9782f1a5b2d2","Type":"ContainerStarted","Data":"ea3e003a638978920f8f8b189ec0a8f0b0265952b1ede8ceb19406e383bfbe3c"} Jul 20 19:38:09.477158 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.477143 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:09.478938 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.478916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" event={"ID":"6954ccee-eaea-402a-8894-49e29808da89","Type":"ContainerStarted","Data":"bc35e3378efb7e5d8868b714977303603aeb86bf0e409565fb298b8375ec92c5"} Jul 20 19:38:09.479056 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.478961 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" event={"ID":"6954ccee-eaea-402a-8894-49e29808da89","Type":"ContainerStarted","Data":"ccbafd0c4fd4227100fe381d667b8a89a3a14bdb5dd6d19937e9db2da32dd31c"} Jul 20 19:38:09.479056 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.478990 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" Jul 20 19:38:09.482641 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.482622 2580 scope.go:117] "RemoveContainer" containerID="d3e2adfd42ae85a250f0da9a7505649a0fe20d837f21b90f316432af4d53d7d8" Jul 20 19:38:09.509557 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.509519 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" podStartSLOduration=1.509504788 podStartE2EDuration="1.509504788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:38:09.508099125 +0000 UTC m=+631.133178119" watchObservedRunningTime="2026-07-20 19:38:09.509504788 +0000 UTC m=+631.134583783" Jul 20 19:38:09.533335 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:09.533290 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" podStartSLOduration=1.533273774 podStartE2EDuration="1.533273774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:38:09.530809047 +0000 UTC m=+631.155888042" watchObservedRunningTime="2026-07-20 19:38:09.533273774 +0000 UTC m=+631.158352767" Jul 20 19:38:20.488138 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:20.488107 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7gkqr" Jul 20 19:38:20.488513 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:20.488280 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:36.269321 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.269281 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd"] Jul 20 19:38:36.269861 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.269508 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" podUID="21676113-5e47-4076-9588-9782f1a5b2d2" containerName="manager" containerID="cri-o://2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6" gracePeriod=10 Jul 20 19:38:36.518476 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.518452 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:36.622572 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.622536 2580 generic.go:356] "Generic (PLEG): container finished" podID="21676113-5e47-4076-9588-9782f1a5b2d2" containerID="2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6" exitCode=0 Jul 20 19:38:36.622770 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.622592 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" Jul 20 19:38:36.622770 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.622619 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" event={"ID":"21676113-5e47-4076-9588-9782f1a5b2d2","Type":"ContainerDied","Data":"2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6"} Jul 20 19:38:36.622770 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.622674 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd" event={"ID":"21676113-5e47-4076-9588-9782f1a5b2d2","Type":"ContainerDied","Data":"ea3e003a638978920f8f8b189ec0a8f0b0265952b1ede8ceb19406e383bfbe3c"} Jul 20 19:38:36.622770 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.622692 2580 scope.go:117] "RemoveContainer" containerID="2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6" Jul 20 19:38:36.631651 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.631617 2580 scope.go:117] "RemoveContainer" containerID="2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6" Jul 20 19:38:36.631902 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:38:36.631885 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6\": container with ID starting with 2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6 not found: ID does not exist" containerID="2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6" Jul 20 19:38:36.631969 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.631920 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6"} err="failed to get container status \"2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6\": rpc error: code = NotFound desc = could not find container \"2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6\": container with ID starting with 2ba13dd8aad9a677a682b1382e2173aebe79e7798074224adadd5bd8af46e1e6 not found: ID does not exist" Jul 20 19:38:36.658253 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.658221 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21676113-5e47-4076-9588-9782f1a5b2d2-extensions-socket-volume\") pod \"21676113-5e47-4076-9588-9782f1a5b2d2\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " Jul 20 19:38:36.658404 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.658267 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxttx\" (UniqueName: \"kubernetes.io/projected/21676113-5e47-4076-9588-9782f1a5b2d2-kube-api-access-mxttx\") pod \"21676113-5e47-4076-9588-9782f1a5b2d2\" (UID: \"21676113-5e47-4076-9588-9782f1a5b2d2\") " Jul 20 19:38:36.658640 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.658614 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21676113-5e47-4076-9588-9782f1a5b2d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "21676113-5e47-4076-9588-9782f1a5b2d2" (UID: "21676113-5e47-4076-9588-9782f1a5b2d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:38:36.660337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.660313 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21676113-5e47-4076-9588-9782f1a5b2d2-kube-api-access-mxttx" (OuterVolumeSpecName: "kube-api-access-mxttx") pod "21676113-5e47-4076-9588-9782f1a5b2d2" (UID: "21676113-5e47-4076-9588-9782f1a5b2d2"). InnerVolumeSpecName "kube-api-access-mxttx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:38:36.759775 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.759732 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21676113-5e47-4076-9588-9782f1a5b2d2-extensions-socket-volume\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:38:36.759775 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.759767 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxttx\" (UniqueName: \"kubernetes.io/projected/21676113-5e47-4076-9588-9782f1a5b2d2-kube-api-access-mxttx\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:38:36.950165 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.950134 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd"] Jul 20 19:38:36.954953 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:36.954923 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5kcqd"] Jul 20 19:38:37.062000 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:37.061969 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21676113-5e47-4076-9588-9782f1a5b2d2" path="/var/lib/kubelet/pods/21676113-5e47-4076-9588-9782f1a5b2d2/volumes" Jul 20 19:38:52.599160 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.599122 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2"] Jul 20 19:38:52.599693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.599676 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21676113-5e47-4076-9588-9782f1a5b2d2" containerName="manager" Jul 20 19:38:52.599739 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.599697 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21676113-5e47-4076-9588-9782f1a5b2d2" containerName="manager" Jul 20 19:38:52.599801 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.599790 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="21676113-5e47-4076-9588-9782f1a5b2d2" containerName="manager" Jul 20 19:38:52.604579 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.604559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.606608 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.606587 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2sp4p\"" Jul 20 19:38:52.620367 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.620327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2"] Jul 20 19:38:52.798997 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.798954 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799199 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799199 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799199 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdh8w\" (UniqueName: \"kubernetes.io/projected/091738fc-368d-424f-9fb1-f5d36400e97c-kube-api-access-fdh8w\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799199 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799199 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091738fc-368d-424f-9fb1-f5d36400e97c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799199 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799527 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799212 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091738fc-368d-424f-9fb1-f5d36400e97c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.799527 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.799248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091738fc-368d-424f-9fb1-f5d36400e97c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900350 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900250 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900350 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdh8w\" (UniqueName: \"kubernetes.io/projected/091738fc-368d-424f-9fb1-f5d36400e97c-kube-api-access-fdh8w\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900350 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900318 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091738fc-368d-424f-9fb1-f5d36400e97c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091738fc-368d-424f-9fb1-f5d36400e97c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091738fc-368d-424f-9fb1-f5d36400e97c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900668 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900955 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900740 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.900955 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.900923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.901237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.901211 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091738fc-368d-424f-9fb1-f5d36400e97c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.901237 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.901224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.901406 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.901226 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.902985 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.902961 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091738fc-368d-424f-9fb1-f5d36400e97c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.903367 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.903347 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091738fc-368d-424f-9fb1-f5d36400e97c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.909781 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.909755 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091738fc-368d-424f-9fb1-f5d36400e97c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.910300 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.910275 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdh8w\" (UniqueName: \"kubernetes.io/projected/091738fc-368d-424f-9fb1-f5d36400e97c-kube-api-access-fdh8w\") pod \"maas-default-gateway-openshift-default-56654dcf7-gqgd2\" (UID: \"091738fc-368d-424f-9fb1-f5d36400e97c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:52.915922 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:52.915895 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:53.066929 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.066906 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2"] Jul 20 19:38:53.069329 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:38:53.069299 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091738fc_368d_424f_9fb1_f5d36400e97c.slice/crio-05ca413bda327dfd779762be282cb650998760832af60a7e374afc6570e677c8 WatchSource:0}: Error finding container 05ca413bda327dfd779762be282cb650998760832af60a7e374afc6570e677c8: Status 404 returned error can't find the container with id 05ca413bda327dfd779762be282cb650998760832af60a7e374afc6570e677c8 Jul 20 19:38:53.071401 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.071371 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:38:53.071518 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.071447 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:38:53.071518 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.071493 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:38:53.716272 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.716239 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" event={"ID":"091738fc-368d-424f-9fb1-f5d36400e97c","Type":"ContainerStarted","Data":"540da9acc5acc60a79cd954438815fad958f62391e5c4fc90cd66e21358f7f13"} Jul 20 19:38:53.716677 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.716300 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" event={"ID":"091738fc-368d-424f-9fb1-f5d36400e97c","Type":"ContainerStarted","Data":"05ca413bda327dfd779762be282cb650998760832af60a7e374afc6570e677c8"} Jul 20 19:38:53.738743 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.738680 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" podStartSLOduration=1.738664543 podStartE2EDuration="1.738664543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:38:53.736311528 +0000 UTC m=+675.361390521" watchObservedRunningTime="2026-07-20 19:38:53.738664543 +0000 UTC m=+675.363743537" Jul 20 19:38:53.916681 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:53.916643 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:54.921609 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:54.921581 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:55.727274 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.727238 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:55.728330 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.728306 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:38:55.854709 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.854669 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:38:55.858765 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.858738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:55.860645 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.860617 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 20 19:38:55.860799 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.860695 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5vdsb\"" Jul 20 19:38:55.866513 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.866169 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:38:55.897191 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.897153 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:38:55.924888 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.924857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a8a809ab-20b8-4554-a765-960d6d810403-config-file\") pod \"limitador-limitador-54d6d988d5-lwnr4\" (UID: \"a8a809ab-20b8-4554-a765-960d6d810403\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:55.925294 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:55.924929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjkmr\" (UniqueName: \"kubernetes.io/projected/a8a809ab-20b8-4554-a765-960d6d810403-kube-api-access-hjkmr\") pod \"limitador-limitador-54d6d988d5-lwnr4\" (UID: \"a8a809ab-20b8-4554-a765-960d6d810403\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:56.025634 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.025533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a8a809ab-20b8-4554-a765-960d6d810403-config-file\") pod \"limitador-limitador-54d6d988d5-lwnr4\" (UID: \"a8a809ab-20b8-4554-a765-960d6d810403\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:56.025634 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.025631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjkmr\" (UniqueName: \"kubernetes.io/projected/a8a809ab-20b8-4554-a765-960d6d810403-kube-api-access-hjkmr\") pod \"limitador-limitador-54d6d988d5-lwnr4\" (UID: \"a8a809ab-20b8-4554-a765-960d6d810403\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:56.026154 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.026135 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a8a809ab-20b8-4554-a765-960d6d810403-config-file\") pod \"limitador-limitador-54d6d988d5-lwnr4\" (UID: \"a8a809ab-20b8-4554-a765-960d6d810403\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:56.032911 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.032884 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjkmr\" (UniqueName: \"kubernetes.io/projected/a8a809ab-20b8-4554-a765-960d6d810403-kube-api-access-hjkmr\") pod \"limitador-limitador-54d6d988d5-lwnr4\" (UID: \"a8a809ab-20b8-4554-a765-960d6d810403\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:56.169687 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.169654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:56.311520 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.311492 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 19:38:56.312182 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.312158 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:38:56.734832 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:56.734746 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" event={"ID":"a8a809ab-20b8-4554-a765-960d6d810403","Type":"ContainerStarted","Data":"2f521801611e9482251bcf89164b278843e91c0985715c29aae24299729fd838"} Jul 20 19:38:59.751505 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:59.751469 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" event={"ID":"a8a809ab-20b8-4554-a765-960d6d810403","Type":"ContainerStarted","Data":"83b4103bffb07921c8f8118d219b8f383cfba0aec522fe5e1f351128bdf038fd"} Jul 20 19:38:59.751923 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:59.751581 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:38:59.769403 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:38:59.769358 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" podStartSLOduration=2.342974051 podStartE2EDuration="4.769343658s" totalImagesPullingTime="2.426369607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:38:59.766385798 +0000 UTC m=+681.391464805" watchObservedRunningTime="2026-07-20 19:38:59.769343658 +0000 UTC m=+681.394422701" Jul 20 19:39:10.757216 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:39:10.757188 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-lwnr4" Jul 20 19:40:14.326980 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:14.326944 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:40:22.186404 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:22.186368 2580 generic.go:356] "Generic (PLEG): container finished" podID="091738fc-368d-424f-9fb1-f5d36400e97c" containerID="540da9acc5acc60a79cd954438815fad958f62391e5c4fc90cd66e21358f7f13" exitCode=0 Jul 20 19:40:22.186847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:22.186463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" event={"ID":"091738fc-368d-424f-9fb1-f5d36400e97c","Type":"ContainerDied","Data":"540da9acc5acc60a79cd954438815fad958f62391e5c4fc90cd66e21358f7f13"} Jul 20 19:40:22.186847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:22.186826 2580 scope.go:117] "RemoveContainer" containerID="540da9acc5acc60a79cd954438815fad958f62391e5c4fc90cd66e21358f7f13" Jul 20 19:40:22.187869 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:22.187840 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:40:22.187980 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:22.187929 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:40:22.188041 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:22.187982 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:40:23.194663 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:23.194608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" event={"ID":"091738fc-368d-424f-9fb1-f5d36400e97c","Type":"ContainerStarted","Data":"8e3f789f494d7e534f79f6417c81c72af8e84c27ba433907465c75d1059bc7cc"} Jul 20 19:40:23.916596 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:23.916560 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:40:23.917773 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:23.917747 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gqgd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 20 19:40:23.917898 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:23.917794 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" podUID="091738fc-368d-424f-9fb1-f5d36400e97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 20 19:40:24.917170 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:24.917134 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gqgd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 20 19:40:24.917592 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:24.917185 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" podUID="091738fc-368d-424f-9fb1-f5d36400e97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 20 19:40:25.727337 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:25.727290 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:40:25.916399 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:25.916361 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gqgd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 20 19:40:25.916602 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:25.916418 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" podUID="091738fc-368d-424f-9fb1-f5d36400e97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 20 19:40:26.916499 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:26.916464 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gqgd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 20 19:40:26.916897 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:26.916521 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" podUID="091738fc-368d-424f-9fb1-f5d36400e97c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 20 19:40:27.920498 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:27.920469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:40:27.921388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:27.921370 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gqgd2" Jul 20 19:40:31.340206 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:31.340164 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:40:41.976400 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:41.976362 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw"] Jul 20 19:40:41.978700 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:41.978677 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:41.981297 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:41.981272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 20 19:40:41.981417 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:41.981275 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 19:40:41.981417 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:41.981291 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bjz7v\"" Jul 20 19:40:41.981417 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:41.981310 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 19:40:41.991413 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:41.991388 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw"] Jul 20 19:40:42.172102 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.172066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.172270 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.172118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c91bbf27-f981-4a4b-98c3-f334fe936d20-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.172270 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.172145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.172270 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.172186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.172270 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.172203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.172455 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.172302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r44f9\" (UniqueName: \"kubernetes.io/projected/c91bbf27-f981-4a4b-98c3-f334fe936d20-kube-api-access-r44f9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273141 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r44f9\" (UniqueName: \"kubernetes.io/projected/c91bbf27-f981-4a4b-98c3-f334fe936d20-kube-api-access-r44f9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273141 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273370 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c91bbf27-f981-4a4b-98c3-f334fe936d20-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273370 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273181 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273370 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273370 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273620 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273595 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273643 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.273680 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.273654 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.275671 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.275643 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c91bbf27-f981-4a4b-98c3-f334fe936d20-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.275840 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.275826 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c91bbf27-f981-4a4b-98c3-f334fe936d20-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.280257 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.280237 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r44f9\" (UniqueName: \"kubernetes.io/projected/c91bbf27-f981-4a4b-98c3-f334fe936d20-kube-api-access-r44f9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vkgfw\" (UID: \"c91bbf27-f981-4a4b-98c3-f334fe936d20\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.291420 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.291398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:42.421220 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.421195 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw"] Jul 20 19:40:42.423661 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:40:42.423632 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91bbf27_f981_4a4b_98c3_f334fe936d20.slice/crio-b76f0959100d7d67c2a24d1bcd1e658cd406acb5a702124275406791ea883bdd WatchSource:0}: Error finding container b76f0959100d7d67c2a24d1bcd1e658cd406acb5a702124275406791ea883bdd: Status 404 returned error can't find the container with id b76f0959100d7d67c2a24d1bcd1e658cd406acb5a702124275406791ea883bdd Jul 20 19:40:42.735973 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:42.735920 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:40:43.300992 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:43.300957 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" event={"ID":"c91bbf27-f981-4a4b-98c3-f334fe936d20","Type":"ContainerStarted","Data":"b76f0959100d7d67c2a24d1bcd1e658cd406acb5a702124275406791ea883bdd"} Jul 20 19:40:44.308182 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:44.308145 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" event={"ID":"c91bbf27-f981-4a4b-98c3-f334fe936d20","Type":"ContainerStarted","Data":"c24848361872a4cc7b8e14a7b947068530c539186905e90d93a40db6c43887b5"} Jul 20 19:40:44.333549 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:44.333496 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" podStartSLOduration=2.350000853 podStartE2EDuration="3.333480186s" totalImagesPullingTime="983.479333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:40:44.331282329 +0000 UTC m=+785.956361321" watchObservedRunningTime="2026-07-20 19:40:44.333480186 +0000 UTC m=+785.958559180" Jul 20 19:40:52.292234 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:52.292194 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:52.292646 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:52.292292 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:52.305053 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:52.305030 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:52.363067 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:52.363033 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vkgfw" Jul 20 19:40:59.478513 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.478481 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4"] Jul 20 19:40:59.484482 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.484458 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.486535 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.486515 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 20 19:40:59.491881 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.491856 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4"] Jul 20 19:40:59.520020 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.519992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.520120 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.520035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ddrv\" (UniqueName: \"kubernetes.io/projected/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-kube-api-access-4ddrv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.520120 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.520095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.520120 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.520117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.520219 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.520139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.520219 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.520156 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.621686 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.621651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.621847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.621697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ddrv\" (UniqueName: \"kubernetes.io/projected/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-kube-api-access-4ddrv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.621847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.621746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.621847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.621778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.621847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.621801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.621847 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.621828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.622226 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.622201 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.622295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.622272 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.622295 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.622279 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.624315 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.624288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.624452 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.624399 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.628913 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.628886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ddrv\" (UniqueName: \"kubernetes.io/projected/dd74c50e-247c-46bc-b2eb-b17cb56b3d5a-kube-api-access-4ddrv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qsfr4\" (UID: \"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.796208 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.796178 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:40:59.927864 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:40:59.927839 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4"] Jul 20 19:40:59.929973 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:40:59.929937 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd74c50e_247c_46bc_b2eb_b17cb56b3d5a.slice/crio-3ec3f678a1754a13228259474a20288c20f00e216ccd904d4500259c1da141bf WatchSource:0}: Error finding container 3ec3f678a1754a13228259474a20288c20f00e216ccd904d4500259c1da141bf: Status 404 returned error can't find the container with id 3ec3f678a1754a13228259474a20288c20f00e216ccd904d4500259c1da141bf Jul 20 19:41:00.396607 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:00.396570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" event={"ID":"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a","Type":"ContainerStarted","Data":"ef2f77107de8f1e9d88c11943826b9249bb55625560cea34c1b8917aafbc8f91"} Jul 20 19:41:00.396771 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:00.396623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" event={"ID":"dd74c50e-247c-46bc-b2eb-b17cb56b3d5a","Type":"ContainerStarted","Data":"3ec3f678a1754a13228259474a20288c20f00e216ccd904d4500259c1da141bf"} Jul 20 19:41:00.431823 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:00.431778 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" podStartSLOduration=1.174978365 podStartE2EDuration="1.431763678s" totalImagesPullingTime="256.785313ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:41:00.429921751 +0000 UTC m=+802.055000745" watchObservedRunningTime="2026-07-20 19:41:00.431763678 +0000 UTC m=+802.056842671" Jul 20 19:41:00.633287 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:00.633249 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:41:09.322081 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:09.322044 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:41:09.796873 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:09.796834 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:41:09.797046 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:09.796909 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:41:09.810027 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:09.809987 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:41:10.459918 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:10.459887 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qsfr4" Jul 20 19:41:23.245293 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:23.245251 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:41:52.026298 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:41:52.026264 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:43:37.883210 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:43:37.883161 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:43:47.485942 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:43:47.485905 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:43:58.572520 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:43:58.572480 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:45:42.579493 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:45:42.579398 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:45:52.141190 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:45:52.141152 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:46:28.471865 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:46:28.471827 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:46:38.360635 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:46:38.360602 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:46:47.573706 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:46:47.573670 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:46:58.533211 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:46:58.533174 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:47:09.939565 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:47:09.939454 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:47:22.441518 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:47:22.441480 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:47:33.369638 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:47:33.369595 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:47:43.553312 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:47:43.553275 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:48:02.142323 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.142287 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76b9f8855f-xn2zq"] Jul 20 19:48:02.145807 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.145788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.196545 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.196510 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b9f8855f-xn2zq"] Jul 20 19:48:02.214622 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.214591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-service-ca\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.214818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.214632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjqwj\" (UniqueName: \"kubernetes.io/projected/da21864d-d850-448c-b254-56ecac205fe7-kube-api-access-gjqwj\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.214818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.214737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da21864d-d850-448c-b254-56ecac205fe7-console-serving-cert\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.214818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.214788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da21864d-d850-448c-b254-56ecac205fe7-console-oauth-config\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.214818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.214815 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-oauth-serving-cert\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.215011 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.214841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-trusted-ca-bundle\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.215011 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.214876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-console-config\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.315619 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.315576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-service-ca\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.315828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.315627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjqwj\" (UniqueName: \"kubernetes.io/projected/da21864d-d850-448c-b254-56ecac205fe7-kube-api-access-gjqwj\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.315828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.315677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da21864d-d850-448c-b254-56ecac205fe7-console-serving-cert\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.315828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.315709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da21864d-d850-448c-b254-56ecac205fe7-console-oauth-config\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.315828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.315771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-oauth-serving-cert\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.315828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.315811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-trusted-ca-bundle\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.316108 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.315845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-console-config\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.316406 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.316378 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-service-ca\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.316797 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.316499 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-oauth-serving-cert\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.316797 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.316649 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-console-config\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.316797 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.316717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da21864d-d850-448c-b254-56ecac205fe7-trusted-ca-bundle\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.318309 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.318284 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da21864d-d850-448c-b254-56ecac205fe7-console-oauth-config\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.318399 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.318360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da21864d-d850-448c-b254-56ecac205fe7-console-serving-cert\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.326284 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.326263 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjqwj\" (UniqueName: \"kubernetes.io/projected/da21864d-d850-448c-b254-56ecac205fe7-kube-api-access-gjqwj\") pod \"console-76b9f8855f-xn2zq\" (UID: \"da21864d-d850-448c-b254-56ecac205fe7\") " pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.455136 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.455045 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:02.594415 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.594389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b9f8855f-xn2zq"] Jul 20 19:48:02.595736 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:48:02.595705 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda21864d_d850_448c_b254_56ecac205fe7.slice/crio-5f24b9249b954f0176a1774390bfe184b4ca33894d8605b9d07fa2edd9e64c2e WatchSource:0}: Error finding container 5f24b9249b954f0176a1774390bfe184b4ca33894d8605b9d07fa2edd9e64c2e: Status 404 returned error can't find the container with id 5f24b9249b954f0176a1774390bfe184b4ca33894d8605b9d07fa2edd9e64c2e Jul 20 19:48:02.623816 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:02.623785 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b9f8855f-xn2zq" event={"ID":"da21864d-d850-448c-b254-56ecac205fe7","Type":"ContainerStarted","Data":"5f24b9249b954f0176a1774390bfe184b4ca33894d8605b9d07fa2edd9e64c2e"} Jul 20 19:48:03.631009 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:03.630972 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b9f8855f-xn2zq" event={"ID":"da21864d-d850-448c-b254-56ecac205fe7","Type":"ContainerStarted","Data":"6dc0681137a146ac84481b31756579de14633932f6dd8533ed8abe3f6f664686"} Jul 20 19:48:03.652608 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:03.652558 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76b9f8855f-xn2zq" podStartSLOduration=1.6525426589999999 podStartE2EDuration="1.652542659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:48:03.649651618 +0000 UTC m=+1225.274730612" watchObservedRunningTime="2026-07-20 19:48:03.652542659 +0000 UTC m=+1225.277621653" Jul 20 19:48:05.534616 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:05.534572 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:48:12.455711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:12.455607 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:12.455711 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:12.455702 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:12.460782 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:12.460753 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:12.685791 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:12.685760 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76b9f8855f-xn2zq" Jul 20 19:48:12.764158 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:12.764061 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c94dfd697-nvzf9"] Jul 20 19:48:16.428564 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:16.428525 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:48:37.786224 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:37.786138 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c94dfd697-nvzf9" podUID="420121bb-5288-4e84-9d0e-baabe508d3d7" containerName="console" containerID="cri-o://4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51" gracePeriod=15 Jul 20 19:48:37.829590 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:37.829554 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:48:38.030417 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.030384 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:48:38.140921 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.140828 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-oauth-serving-cert\") pod \"420121bb-5288-4e84-9d0e-baabe508d3d7\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " Jul 20 19:48:38.140921 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.140865 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-oauth-config\") pod \"420121bb-5288-4e84-9d0e-baabe508d3d7\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " Jul 20 19:48:38.140921 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.140902 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-trusted-ca-bundle\") pod \"420121bb-5288-4e84-9d0e-baabe508d3d7\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " Jul 20 19:48:38.141215 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.140925 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-serving-cert\") pod \"420121bb-5288-4e84-9d0e-baabe508d3d7\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " Jul 20 19:48:38.141215 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.140989 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-service-ca\") pod \"420121bb-5288-4e84-9d0e-baabe508d3d7\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " Jul 20 19:48:38.141215 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.141040 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-console-config\") pod \"420121bb-5288-4e84-9d0e-baabe508d3d7\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " Jul 20 19:48:38.141215 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.141063 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhp9j\" (UniqueName: \"kubernetes.io/projected/420121bb-5288-4e84-9d0e-baabe508d3d7-kube-api-access-rhp9j\") pod \"420121bb-5288-4e84-9d0e-baabe508d3d7\" (UID: \"420121bb-5288-4e84-9d0e-baabe508d3d7\") " Jul 20 19:48:38.141421 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.141388 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "420121bb-5288-4e84-9d0e-baabe508d3d7" (UID: "420121bb-5288-4e84-9d0e-baabe508d3d7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:48:38.141503 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.141414 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-service-ca" (OuterVolumeSpecName: "service-ca") pod "420121bb-5288-4e84-9d0e-baabe508d3d7" (UID: "420121bb-5288-4e84-9d0e-baabe508d3d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:48:38.141503 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.141466 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "420121bb-5288-4e84-9d0e-baabe508d3d7" (UID: "420121bb-5288-4e84-9d0e-baabe508d3d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:48:38.141577 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.141556 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-console-config" (OuterVolumeSpecName: "console-config") pod "420121bb-5288-4e84-9d0e-baabe508d3d7" (UID: "420121bb-5288-4e84-9d0e-baabe508d3d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:48:38.143203 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.143169 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/420121bb-5288-4e84-9d0e-baabe508d3d7-kube-api-access-rhp9j" (OuterVolumeSpecName: "kube-api-access-rhp9j") pod "420121bb-5288-4e84-9d0e-baabe508d3d7" (UID: "420121bb-5288-4e84-9d0e-baabe508d3d7"). InnerVolumeSpecName "kube-api-access-rhp9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:48:38.143312 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.143205 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "420121bb-5288-4e84-9d0e-baabe508d3d7" (UID: "420121bb-5288-4e84-9d0e-baabe508d3d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:48:38.143312 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.143249 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "420121bb-5288-4e84-9d0e-baabe508d3d7" (UID: "420121bb-5288-4e84-9d0e-baabe508d3d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:48:38.241938 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.241901 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-trusted-ca-bundle\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:48:38.241938 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.241932 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-serving-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:48:38.241938 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.241942 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-service-ca\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:48:38.242177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.241951 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-console-config\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:48:38.242177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.241960 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhp9j\" (UniqueName: \"kubernetes.io/projected/420121bb-5288-4e84-9d0e-baabe508d3d7-kube-api-access-rhp9j\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:48:38.242177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.241969 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/420121bb-5288-4e84-9d0e-baabe508d3d7-oauth-serving-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:48:38.242177 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.241977 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/420121bb-5288-4e84-9d0e-baabe508d3d7-console-oauth-config\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:48:38.819541 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.819506 2580 generic.go:356] "Generic (PLEG): container finished" podID="420121bb-5288-4e84-9d0e-baabe508d3d7" containerID="4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51" exitCode=0 Jul 20 19:48:38.820011 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.819547 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c94dfd697-nvzf9" Jul 20 19:48:38.820011 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.819588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c94dfd697-nvzf9" event={"ID":"420121bb-5288-4e84-9d0e-baabe508d3d7","Type":"ContainerDied","Data":"4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51"} Jul 20 19:48:38.820011 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.819637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c94dfd697-nvzf9" event={"ID":"420121bb-5288-4e84-9d0e-baabe508d3d7","Type":"ContainerDied","Data":"0012f34962250d1727342fe6645c56599066f0b7d00f49ec9771fc799241d0b9"} Jul 20 19:48:38.820011 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.819659 2580 scope.go:117] "RemoveContainer" containerID="4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51" Jul 20 19:48:38.833800 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.833776 2580 scope.go:117] "RemoveContainer" containerID="4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51" Jul 20 19:48:38.834089 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:48:38.834068 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51\": container with ID starting with 4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51 not found: ID does not exist" containerID="4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51" Jul 20 19:48:38.834141 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.834097 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51"} err="failed to get container status \"4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51\": rpc error: code = NotFound desc = could not find container \"4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51\": container with ID starting with 4665b07741d3fd1f47b0bd811a4ecc7f28456e6f368677352bebc8ef1a2a2e51 not found: ID does not exist" Jul 20 19:48:38.848362 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.848337 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c94dfd697-nvzf9"] Jul 20 19:48:38.854871 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:38.854843 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c94dfd697-nvzf9"] Jul 20 19:48:39.062138 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:39.062105 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="420121bb-5288-4e84-9d0e-baabe508d3d7" path="/var/lib/kubelet/pods/420121bb-5288-4e84-9d0e-baabe508d3d7/volumes" Jul 20 19:48:51.233799 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:48:51.233747 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:49:04.830019 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:49:04.829978 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:49:16.329492 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:49:16.329444 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:49:26.230165 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:49:26.230130 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:49:36.029085 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:49:36.029046 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:49:45.560578 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:49:45.560547 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:51:31.333496 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:51:31.333394 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:51:40.936029 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:51:40.935997 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:51:54.366844 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:51:54.366807 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:52:05.133084 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:52:05.133044 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:52:15.948809 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:52:15.948771 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:52:27.247842 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:52:27.247803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:52:37.132787 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:52:37.132751 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:52:48.031262 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:52:48.031227 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:52:59.635937 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:52:59.635897 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:53:07.850367 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:53:07.850272 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:53:18.942163 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:53:18.942129 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:53:45.361290 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:53:45.361251 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:53:52.350215 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:53:52.350172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:54:04.121449 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:54:04.121391 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:54:28.852647 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:54:28.852617 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:55:03.732707 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:55:03.732667 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:55:19.753158 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:55:19.753114 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:55:26.526251 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:55:26.526217 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:55:42.434878 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:55:42.434842 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:55:53.353208 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:55:53.353167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:56:02.929819 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:56:02.929737 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:56:13.634553 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:56:13.634514 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:56:22.846458 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:56:22.846409 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:56:32.335166 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:56:32.335126 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:56:45.850277 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:56:45.850236 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:56:55.632828 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:56:55.632785 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:57:05.440101 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:57:05.440055 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:57:18.232695 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:57:18.232656 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:57:29.542468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:57:29.542418 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:57:38.635388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:57:38.635348 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:57:50.832905 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:57:50.832870 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:58:03.233180 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:03.233140 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:58:16.330041 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:16.330005 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:58:25.126554 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:25.126522 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:58:38.045989 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:38.045953 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:58:46.767004 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:46.766957 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:58:53.052301 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.052265 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn"] Jul 20 19:58:53.053398 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.053371 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="420121bb-5288-4e84-9d0e-baabe508d3d7" containerName="console" Jul 20 19:58:53.053398 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.053399 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="420121bb-5288-4e84-9d0e-baabe508d3d7" containerName="console" Jul 20 19:58:53.053555 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.053499 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="420121bb-5288-4e84-9d0e-baabe508d3d7" containerName="console" Jul 20 19:58:53.056937 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.056913 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.059674 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.059650 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4f42155d-openshift-default-dockercfg-b4n2p\"" Jul 20 19:58:53.076622 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.076596 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn"] Jul 20 19:58:53.155576 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155541 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h62z\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-kube-api-access-6h62z\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.155767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-data\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.155767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-podinfo\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.155767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-socket\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.155767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-certs\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.155767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1435d79e-f67f-4fbd-8229-274bdf7fef46-istiod-ca-cert\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.155767 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-envoy\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.156100 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-token\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.156100 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.155917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-credential-socket\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.256950 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.256917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-data\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.256954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-podinfo\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.256979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-socket\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-certs\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1435d79e-f67f-4fbd-8229-274bdf7fef46-istiod-ca-cert\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257116 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-envoy\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257371 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257130 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-token\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257371 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-credential-socket\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257371 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h62z\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-kube-api-access-6h62z\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257557 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257476 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-certs\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257557 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257530 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-credential-socket\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257664 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-data\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257953 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257908 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-socket\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.257953 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.257916 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1435d79e-f67f-4fbd-8229-274bdf7fef46-istiod-ca-cert\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.259600 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.259568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-podinfo\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.259685 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.259602 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-envoy\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.264868 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.264836 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h62z\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-kube-api-access-6h62z\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.265153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.265134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-token\") pod \"e2e-ait-4f42155d-openshift-default-5989664f-cxgbn\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.369491 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.369366 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:53.510551 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.510526 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn"] Jul 20 19:58:53.512032 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:58:53.512004 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1435d79e_f67f_4fbd_8229_274bdf7fef46.slice/crio-166171a07da2c5ab69b81419ed9589f14130a8f5522bc47b6831504076484948 WatchSource:0}: Error finding container 166171a07da2c5ab69b81419ed9589f14130a8f5522bc47b6831504076484948: Status 404 returned error can't find the container with id 166171a07da2c5ab69b81419ed9589f14130a8f5522bc47b6831504076484948 Jul 20 19:58:53.514132 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.514099 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:58:53.514218 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.514161 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:58:53.514218 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:53.514190 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:58:54.057221 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:54.057187 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" event={"ID":"1435d79e-f67f-4fbd-8229-274bdf7fef46","Type":"ContainerStarted","Data":"034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c"} Jul 20 19:58:54.057641 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:54.057237 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" event={"ID":"1435d79e-f67f-4fbd-8229-274bdf7fef46","Type":"ContainerStarted","Data":"166171a07da2c5ab69b81419ed9589f14130a8f5522bc47b6831504076484948"} Jul 20 19:58:54.079637 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:54.079589 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" podStartSLOduration=1.079573685 podStartE2EDuration="1.079573685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:58:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:58:54.077550977 +0000 UTC m=+1875.702629970" watchObservedRunningTime="2026-07-20 19:58:54.079573685 +0000 UTC m=+1875.704652747" Jul 20 19:58:54.369783 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:54.369686 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:54.375009 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:54.374984 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:55.065238 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:55.065206 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:58:55.066273 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:58:55.066252 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:59:02.033077 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:02.032993 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:59:10.066411 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.066376 2580 patch_prober.go:28] interesting pod/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 20 19:59:10.066927 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.066462 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 20 19:59:10.145125 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.145026 2580 generic.go:356] "Generic (PLEG): container finished" podID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerID="034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c" exitCode=0 Jul 20 19:59:10.145125 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.145104 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" event={"ID":"1435d79e-f67f-4fbd-8229-274bdf7fef46","Type":"ContainerDied","Data":"034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c"} Jul 20 19:59:10.145560 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.145543 2580 scope.go:117] "RemoveContainer" containerID="034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c" Jul 20 19:59:10.146589 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.146563 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:10.146693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.146631 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:10.146693 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:10.146660 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:11.153932 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:11.153895 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" event={"ID":"1435d79e-f67f-4fbd-8229-274bdf7fef46","Type":"ContainerStarted","Data":"e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754"} Jul 20 19:59:11.370639 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:11.370596 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:59:11.370832 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:11.370812 2580 patch_prober.go:28] interesting pod/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 20 19:59:11.370895 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:11.370876 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 20 19:59:12.370161 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:12.370127 2580 patch_prober.go:28] interesting pod/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 20 19:59:12.370565 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:12.370189 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 20 19:59:13.370619 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:13.370572 2580 patch_prober.go:28] interesting pod/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 20 19:59:13.371012 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:13.370668 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 20 19:59:14.370221 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:14.370180 2580 patch_prober.go:28] interesting pod/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 20 19:59:14.370407 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:14.370256 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 20 19:59:15.374418 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:15.374387 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:59:15.374804 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:15.374757 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:59:15.375412 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:15.375394 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:59:34.178197 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:34.178157 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:59:36.350093 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:36.350053 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn"] Jul 20 19:59:36.350512 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:36.350314 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" containerID="cri-o://e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754" gracePeriod=30 Jul 20 19:59:37.594814 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.594774 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7"] Jul 20 19:59:37.597807 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.597783 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.604021 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.603992 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a2049ed7-gw-openshift-default-dockercfg-zl8dt\"" Jul 20 19:59:37.629301 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.629253 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7"] Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684149 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-data\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g4mj\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-kube-api-access-2g4mj\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-token\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-certs\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-credential-socket\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684338 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-socket\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689d8f5f-f49d-4a12-8abf-06048a953d2d-istiod-ca-cert\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-podinfo\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.684649 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.684507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-envoy\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785218 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-credential-socket\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785423 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-socket\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785423 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689d8f5f-f49d-4a12-8abf-06048a953d2d-istiod-ca-cert\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785423 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-podinfo\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785423 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-envoy\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785694 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-data\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785694 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g4mj\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-kube-api-access-2g4mj\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785694 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-token\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785694 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785574 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-certs\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785694 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785675 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-credential-socket\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.785938 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.785730 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-socket\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.786093 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.786073 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689d8f5f-f49d-4a12-8abf-06048a953d2d-istiod-ca-cert\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.786165 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.786123 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-certs\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.786221 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.786178 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-data\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.787835 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.787817 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-envoy\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.787980 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.787962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-podinfo\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.793359 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.793338 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-token\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.793468 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.793370 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g4mj\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-kube-api-access-2g4mj\") pod \"e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:37.908626 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:37.908530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:38.058142 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.058112 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7"] Jul 20 19:59:38.060154 ip-10-0-132-219 kubenswrapper[2580]: W0720 19:59:38.060123 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod689d8f5f_f49d_4a12_8abf_06048a953d2d.slice/crio-5a8ab8e896843438a4c0935d5797cb78c4d2093456008259fcc6279806ef472e WatchSource:0}: Error finding container 5a8ab8e896843438a4c0935d5797cb78c4d2093456008259fcc6279806ef472e: Status 404 returned error can't find the container with id 5a8ab8e896843438a4c0935d5797cb78c4d2093456008259fcc6279806ef472e Jul 20 19:59:38.062647 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.062613 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:38.062755 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.062683 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:38.062755 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.062715 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:38.305389 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.305351 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" event={"ID":"689d8f5f-f49d-4a12-8abf-06048a953d2d","Type":"ContainerStarted","Data":"ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25"} Jul 20 19:59:38.305613 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.305404 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" event={"ID":"689d8f5f-f49d-4a12-8abf-06048a953d2d","Type":"ContainerStarted","Data":"5a8ab8e896843438a4c0935d5797cb78c4d2093456008259fcc6279806ef472e"} Jul 20 19:59:38.329636 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.329573 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" podStartSLOduration=1.3295499689999999 podStartE2EDuration="1.329549969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:59:38.326582412 +0000 UTC m=+1919.951661411" watchObservedRunningTime="2026-07-20 19:59:38.329549969 +0000 UTC m=+1919.954628965" Jul 20 19:59:38.909440 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:38.909394 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:39.914032 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:39.913998 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:40.316274 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:40.316238 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:40.317160 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:40.317139 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:41.605699 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.605674 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:59:41.621783 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621707 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-podinfo\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.621783 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621760 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h62z\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-kube-api-access-6h62z\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.621985 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621788 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-certs\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.621985 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621811 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-token\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.621985 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621865 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-data\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.621985 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621909 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-socket\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.621985 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621932 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-envoy\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.622233 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.621983 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-credential-socket\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.622233 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.622072 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1435d79e-f67f-4fbd-8229-274bdf7fef46-istiod-ca-cert\") pod \"1435d79e-f67f-4fbd-8229-274bdf7fef46\" (UID: \"1435d79e-f67f-4fbd-8229-274bdf7fef46\") " Jul 20 19:59:41.622233 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.622112 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:59:41.622388 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.622357 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:59:41.622469 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.622414 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:59:41.622469 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.622420 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.622759 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.622738 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1435d79e-f67f-4fbd-8229-274bdf7fef46-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:59:41.623189 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.623158 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-data" (OuterVolumeSpecName: "istio-data") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:59:41.624926 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.624901 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-token" (OuterVolumeSpecName: "istio-token") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:59:41.624926 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.624914 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:59:41.625103 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.624933 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-kube-api-access-6h62z" (OuterVolumeSpecName: "kube-api-access-6h62z") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "kube-api-access-6h62z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:59:41.625103 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.624996 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1435d79e-f67f-4fbd-8229-274bdf7fef46" (UID: "1435d79e-f67f-4fbd-8229-274bdf7fef46"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 19:59:41.723780 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723737 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.723780 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723774 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h62z\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-kube-api-access-6h62z\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.723780 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723788 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.724105 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723800 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.724105 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723813 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.724105 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723824 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.724105 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723835 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1435d79e-f67f-4fbd-8229-274bdf7fef46-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:41.724105 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:41.723854 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1435d79e-f67f-4fbd-8229-274bdf7fef46-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 19:59:42.328643 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.328601 2580 generic.go:356] "Generic (PLEG): container finished" podID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerID="e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754" exitCode=0 Jul 20 19:59:42.328818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.328656 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" Jul 20 19:59:42.328818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.328690 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" event={"ID":"1435d79e-f67f-4fbd-8229-274bdf7fef46","Type":"ContainerDied","Data":"e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754"} Jul 20 19:59:42.328818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.328756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn" event={"ID":"1435d79e-f67f-4fbd-8229-274bdf7fef46","Type":"ContainerDied","Data":"166171a07da2c5ab69b81419ed9589f14130a8f5522bc47b6831504076484948"} Jul 20 19:59:42.328818 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.328798 2580 scope.go:117] "RemoveContainer" containerID="e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754" Jul 20 19:59:42.339086 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.339069 2580 scope.go:117] "RemoveContainer" containerID="034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c" Jul 20 19:59:42.350301 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.350276 2580 scope.go:117] "RemoveContainer" containerID="e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754" Jul 20 19:59:42.350606 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:59:42.350584 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754\": container with ID starting with e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754 not found: ID does not exist" containerID="e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754" Jul 20 19:59:42.350702 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.350614 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754"} err="failed to get container status \"e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754\": rpc error: code = NotFound desc = could not find container \"e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754\": container with ID starting with e26ee722b842bce909806e870dc9ed12e3bca6bacdc78ff79bcaeb69119c5754 not found: ID does not exist" Jul 20 19:59:42.350702 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.350632 2580 scope.go:117] "RemoveContainer" containerID="034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c" Jul 20 19:59:42.350918 ip-10-0-132-219 kubenswrapper[2580]: E0720 19:59:42.350898 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c\": container with ID starting with 034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c not found: ID does not exist" containerID="034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c" Jul 20 19:59:42.350961 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.350924 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c"} err="failed to get container status \"034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c\": rpc error: code = NotFound desc = could not find container \"034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c\": container with ID starting with 034ecf90a105a810d01ba057bf9e0c15a335f417ecf46d7751e4f1b39907b50c not found: ID does not exist" Jul 20 19:59:42.352341 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.352283 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn"] Jul 20 19:59:42.355540 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:42.355518 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4f42155d-openshift-default-5989664f-cxgbn"] Jul 20 19:59:43.062414 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:43.062382 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" path="/var/lib/kubelet/pods/1435d79e-f67f-4fbd-8229-274bdf7fef46/volumes" Jul 20 19:59:47.381636 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:47.381597 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 19:59:55.317466 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.317410 2580 patch_prober.go:28] interesting pod/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 19:59:55.317911 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.317495 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 19:59:55.401722 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.401692 2580 generic.go:356] "Generic (PLEG): container finished" podID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerID="ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25" exitCode=0 Jul 20 19:59:55.401909 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.401758 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" event={"ID":"689d8f5f-f49d-4a12-8abf-06048a953d2d","Type":"ContainerDied","Data":"ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25"} Jul 20 19:59:55.402218 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.402201 2580 scope.go:117] "RemoveContainer" containerID="ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25" Jul 20 19:59:55.403299 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.403273 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:55.403371 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.403354 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:55.403409 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:55.403399 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 19:59:56.409777 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:56.409740 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" event={"ID":"689d8f5f-f49d-4a12-8abf-06048a953d2d","Type":"ContainerStarted","Data":"807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739"} Jul 20 19:59:56.908669 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:56.908637 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 19:59:56.908843 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:56.908787 2580 patch_prober.go:28] interesting pod/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 19:59:56.908892 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:56.908866 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 19:59:57.909741 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:57.909704 2580 patch_prober.go:28] interesting pod/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 19:59:57.910153 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:57.909766 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 19:59:58.909841 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:58.909805 2580 patch_prober.go:28] interesting pod/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 19:59:58.910245 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:58.909862 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 19:59:59.909226 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:59.909188 2580 patch_prober.go:28] interesting pod/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 20 19:59:59.909484 ip-10-0-132-219 kubenswrapper[2580]: I0720 19:59:59.909249 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 20 20:00:00.912972 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:00.912941 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 20:00:00.913399 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:00.913298 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 20:00:00.913867 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:00.913848 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 20:00:18.220883 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:18.220849 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:00:20.173707 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:20.173667 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7"] Jul 20 20:00:20.174148 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:20.173896 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" containerID="cri-o://807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739" gracePeriod=30 Jul 20 20:00:21.012752 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.012717 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr"] Jul 20 20:00:21.013181 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.013163 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" Jul 20 20:00:21.013279 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.013184 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" Jul 20 20:00:21.013279 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.013200 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" Jul 20 20:00:21.013279 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.013208 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" Jul 20 20:00:21.013464 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.013293 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" Jul 20 20:00:21.013464 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.013306 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1435d79e-f67f-4fbd-8229-274bdf7fef46" containerName="istio-proxy" Jul 20 20:00:21.015775 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.015753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.018243 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.018220 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8b313d06-openshift-default-dockercfg-2jxvk\"" Jul 20 20:00:21.031700 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.031672 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr"] Jul 20 20:00:21.188007 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.187964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-credential-socket\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188007 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-envoy\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-socket\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188194 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-certs\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-token\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-podinfo\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x6z9\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-kube-api-access-4x6z9\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istiod-ca-cert\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.188493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.188404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-data\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.289580 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-certs\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.289777 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-token\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.289777 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-podinfo\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.289777 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289683 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x6z9\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-kube-api-access-4x6z9\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.289777 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istiod-ca-cert\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.289777 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-data\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289787 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-credential-socket\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-envoy\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.289922 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-socket\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290190 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.290042 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-certs\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290190 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.290153 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-data\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290303 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.290260 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-socket\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290303 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.290262 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-credential-socket\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.290598 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.290575 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istiod-ca-cert\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.292158 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.292134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-envoy\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.292314 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.292297 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-podinfo\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.297905 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.297884 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-token\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.298190 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.298169 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x6z9\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-kube-api-access-4x6z9\") pod \"e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.326113 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.326074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:21.466309 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.466280 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr"] Jul 20 20:00:21.467116 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:00:21.467090 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81ee9d5_06e3_469a_a1e4_08d0cbc09754.slice/crio-9cd15d209a2bfdea6c0a2f5560605305610dfe9d8da6a4fd2b76c08ff3dc9cc8 WatchSource:0}: Error finding container 9cd15d209a2bfdea6c0a2f5560605305610dfe9d8da6a4fd2b76c08ff3dc9cc8: Status 404 returned error can't find the container with id 9cd15d209a2bfdea6c0a2f5560605305610dfe9d8da6a4fd2b76c08ff3dc9cc8 Jul 20 20:00:21.469328 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.469290 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:00:21.469494 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.469355 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:00:21.469494 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.469399 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:00:21.548920 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.548823 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" event={"ID":"e81ee9d5-06e3-469a-a1e4-08d0cbc09754","Type":"ContainerStarted","Data":"90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639"} Jul 20 20:00:21.548920 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.548905 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" event={"ID":"e81ee9d5-06e3-469a-a1e4-08d0cbc09754","Type":"ContainerStarted","Data":"9cd15d209a2bfdea6c0a2f5560605305610dfe9d8da6a4fd2b76c08ff3dc9cc8"} Jul 20 20:00:21.569928 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:21.569866 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" podStartSLOduration=1.569843805 podStartE2EDuration="1.569843805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:00:21.566471846 +0000 UTC m=+1963.191550851" watchObservedRunningTime="2026-07-20 20:00:21.569843805 +0000 UTC m=+1963.194922800" Jul 20 20:00:22.326515 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:22.326476 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:22.331982 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:22.331953 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:22.554394 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:22.554367 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:22.555481 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:22.555460 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:25.428023 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.427991 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 20:00:25.531220 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531132 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-certs\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531220 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531172 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-socket\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531220 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531213 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-token\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531244 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-envoy\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531283 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-podinfo\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531304 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2g4mj\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-kube-api-access-2g4mj\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531329 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-credential-socket\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531354 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-data\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531376 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689d8f5f-f49d-4a12-8abf-06048a953d2d-istiod-ca-cert\") pod \"689d8f5f-f49d-4a12-8abf-06048a953d2d\" (UID: \"689d8f5f-f49d-4a12-8abf-06048a953d2d\") " Jul 20 20:00:25.531549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531496 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:00:25.531878 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531587 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:00:25.531878 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531766 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.531878 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531787 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.531996 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.531887 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/689d8f5f-f49d-4a12-8abf-06048a953d2d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:00:25.532047 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.532022 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:00:25.532602 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.532569 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-data" (OuterVolumeSpecName: "istio-data") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:00:25.533686 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.533659 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:00:25.533793 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.533740 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:00:25.533836 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.533795 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-kube-api-access-2g4mj" (OuterVolumeSpecName: "kube-api-access-2g4mj") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "kube-api-access-2g4mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:00:25.533836 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.533796 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "689d8f5f-f49d-4a12-8abf-06048a953d2d" (UID: "689d8f5f-f49d-4a12-8abf-06048a953d2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:00:25.572615 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.572579 2580 generic.go:356] "Generic (PLEG): container finished" podID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerID="807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739" exitCode=0 Jul 20 20:00:25.572785 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.572638 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" Jul 20 20:00:25.572785 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.572667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" event={"ID":"689d8f5f-f49d-4a12-8abf-06048a953d2d","Type":"ContainerDied","Data":"807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739"} Jul 20 20:00:25.572785 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.572729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7" event={"ID":"689d8f5f-f49d-4a12-8abf-06048a953d2d","Type":"ContainerDied","Data":"5a8ab8e896843438a4c0935d5797cb78c4d2093456008259fcc6279806ef472e"} Jul 20 20:00:25.572785 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.572755 2580 scope.go:117] "RemoveContainer" containerID="807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739" Jul 20 20:00:25.582578 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.582561 2580 scope.go:117] "RemoveContainer" containerID="ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25" Jul 20 20:00:25.590756 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.590737 2580 scope.go:117] "RemoveContainer" containerID="807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739" Jul 20 20:00:25.591039 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:00:25.591018 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739\": container with ID starting with 807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739 not found: ID does not exist" containerID="807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739" Jul 20 20:00:25.591087 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.591049 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739"} err="failed to get container status \"807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739\": rpc error: code = NotFound desc = could not find container \"807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739\": container with ID starting with 807103788b630806cac147ccefbb354a46104785141e063927ef5f9f77e14739 not found: ID does not exist" Jul 20 20:00:25.591087 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.591070 2580 scope.go:117] "RemoveContainer" containerID="ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25" Jul 20 20:00:25.591456 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:00:25.591395 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25\": container with ID starting with ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25 not found: ID does not exist" containerID="ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25" Jul 20 20:00:25.591456 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.591447 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25"} err="failed to get container status \"ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25\": rpc error: code = NotFound desc = could not find container \"ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25\": container with ID starting with ae751e513341bb78cf22f3451f41207e195e22360783a6b6582542bd19d97b25 not found: ID does not exist" Jul 20 20:00:25.594577 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.594553 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7"] Jul 20 20:00:25.597384 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.597360 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a2049ed7-gw-openshift-default-6b7f84ddbc-mmgk7"] Jul 20 20:00:25.632337 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.632311 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g4mj\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-kube-api-access-2g4mj\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.632337 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.632333 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.632337 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.632344 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.632548 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.632353 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/689d8f5f-f49d-4a12-8abf-06048a953d2d-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.632548 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.632361 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.632548 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.632370 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:25.632548 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:25.632377 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/689d8f5f-f49d-4a12-8abf-06048a953d2d-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:00:27.061772 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:27.061740 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" path="/var/lib/kubelet/pods/689d8f5f-f49d-4a12-8abf-06048a953d2d/volumes" Jul 20 20:00:29.337200 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:29.337158 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:00:37.556125 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.556087 2580 patch_prober.go:28] interesting pod/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 20 20:00:37.556663 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.556165 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 20 20:00:37.645792 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.645753 2580 generic.go:356] "Generic (PLEG): container finished" podID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerID="90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639" exitCode=0 Jul 20 20:00:37.645975 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.645825 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" event={"ID":"e81ee9d5-06e3-469a-a1e4-08d0cbc09754","Type":"ContainerDied","Data":"90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639"} Jul 20 20:00:37.646251 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.646222 2580 scope.go:117] "RemoveContainer" containerID="90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639" Jul 20 20:00:37.647233 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.647205 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:00:37.647330 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.647272 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:00:37.647330 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:37.647306 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:00:38.654605 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:38.654572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" event={"ID":"e81ee9d5-06e3-469a-a1e4-08d0cbc09754","Type":"ContainerStarted","Data":"8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a"} Jul 20 20:00:39.326667 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:39.326625 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:39.327529 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:39.327214 2580 patch_prober.go:28] interesting pod/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 20 20:00:39.327529 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:39.327269 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 20 20:00:40.327618 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:40.327578 2580 patch_prober.go:28] interesting pod/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 20 20:00:40.328225 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:40.328189 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 20 20:00:41.327055 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:41.327012 2580 patch_prober.go:28] interesting pod/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 20 20:00:41.327285 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:41.327083 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 20 20:00:42.334870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:42.334824 2580 patch_prober.go:28] interesting pod/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 20 20:00:42.335369 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:42.334895 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 20 20:00:43.330062 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:43.330029 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:43.330477 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:43.330452 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:00:43.331091 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:00:43.331075 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:01:02.126979 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:02.126941 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:01:05.221491 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:05.221453 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr"] Jul 20 20:01:05.222003 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:05.221688 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" containerID="cri-o://8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a" gracePeriod=30 Jul 20 20:01:06.129254 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.129213 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7"] Jul 20 20:01:06.129696 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.129676 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" Jul 20 20:01:06.129814 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.129698 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" Jul 20 20:01:06.129814 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.129721 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" Jul 20 20:01:06.129814 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.129730 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" Jul 20 20:01:06.129973 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.129823 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" Jul 20 20:01:06.133213 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.133190 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.135277 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.135253 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-06dcaf8a-openshift-default-dockercfg-vsgwn\"" Jul 20 20:01:06.148457 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.148414 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7"] Jul 20 20:01:06.218856 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.218818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-podinfo\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219041 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.218874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dtkz\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-kube-api-access-7dtkz\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219041 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.218972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istiod-ca-cert\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219121 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.219047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-socket\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219121 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.219083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-data\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219219 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.219195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-envoy\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219261 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.219246 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-certs\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219296 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.219284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-token\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.219330 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.219314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-credential-socket\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320335 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istiod-ca-cert\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320335 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320343 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-socket\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320362 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-data\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-envoy\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-certs\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-token\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320816 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-socket\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-data\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.320894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-credential-socket\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.321258 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320933 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-certs\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.321258 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320950 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-podinfo\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.321258 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.320989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dtkz\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-kube-api-access-7dtkz\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.321258 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.321093 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-credential-socket\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.321258 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.321121 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istiod-ca-cert\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.322839 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.322813 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-envoy\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.323278 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.323256 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-podinfo\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.328606 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.328569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dtkz\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-kube-api-access-7dtkz\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.331940 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.331917 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-token\") pod \"e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.444329 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.444225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:06.586756 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.586728 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7"] Jul 20 20:01:06.587884 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:01:06.587846 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2d03c2_6686_4b34_9c28_b9c5c8cb0905.slice/crio-ff6f621826d021ecc1e364c9b09da7a58138264505e525b1f7fd4a4c4d6496af WatchSource:0}: Error finding container ff6f621826d021ecc1e364c9b09da7a58138264505e525b1f7fd4a4c4d6496af: Status 404 returned error can't find the container with id ff6f621826d021ecc1e364c9b09da7a58138264505e525b1f7fd4a4c4d6496af Jul 20 20:01:06.589971 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.589937 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:01:06.590051 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.590018 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:01:06.590091 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.590048 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:01:06.807900 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.807858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" event={"ID":"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905","Type":"ContainerStarted","Data":"9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9"} Jul 20 20:01:06.808081 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.807925 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" event={"ID":"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905","Type":"ContainerStarted","Data":"ff6f621826d021ecc1e364c9b09da7a58138264505e525b1f7fd4a4c4d6496af"} Jul 20 20:01:06.830013 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:06.829959 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" podStartSLOduration=0.829924276 podStartE2EDuration="829.924276ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:01:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:01:06.828262907 +0000 UTC m=+2008.453341913" watchObservedRunningTime="2026-07-20 20:01:06.829924276 +0000 UTC m=+2008.455003272" Jul 20 20:01:07.444928 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:07.444891 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:07.449878 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:07.449851 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:07.813578 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:07.813548 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:07.814368 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:07.814344 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:10.475324 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.475301 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:01:10.563766 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.563739 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x6z9\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-kube-api-access-4x6z9\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.563915 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.563802 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-credential-socket\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.563960 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.563912 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-certs\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.563960 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.563952 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-podinfo\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.564025 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.563985 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-token\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.564025 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564007 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istiod-ca-cert\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.564106 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564025 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-socket\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.564106 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564049 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-data\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.564106 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564060 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:10.564106 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564084 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-envoy\") pod \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\" (UID: \"e81ee9d5-06e3-469a-a1e4-08d0cbc09754\") " Jul 20 20:01:10.564306 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564188 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:10.564367 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564306 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:10.564463 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564386 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:01:10.564636 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564614 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.564723 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564642 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.564723 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564656 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.564723 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564669 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.564975 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.564950 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-data" (OuterVolumeSpecName: "istio-data") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:10.566214 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.566193 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-token" (OuterVolumeSpecName: "istio-token") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:01:10.566336 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.566243 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:01:10.566453 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.566417 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-kube-api-access-4x6z9" (OuterVolumeSpecName: "kube-api-access-4x6z9") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "kube-api-access-4x6z9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:01:10.566562 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.566546 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e81ee9d5-06e3-469a-a1e4-08d0cbc09754" (UID: "e81ee9d5-06e3-469a-a1e4-08d0cbc09754"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:10.665804 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.665776 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x6z9\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-kube-api-access-4x6z9\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.665804 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.665801 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.665804 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.665810 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.666006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.665819 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.666006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.665828 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e81ee9d5-06e3-469a-a1e4-08d0cbc09754-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:10.830090 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.830001 2580 generic.go:356] "Generic (PLEG): container finished" podID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerID="8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a" exitCode=0 Jul 20 20:01:10.830090 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.830058 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" Jul 20 20:01:10.830090 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.830079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" event={"ID":"e81ee9d5-06e3-469a-a1e4-08d0cbc09754","Type":"ContainerDied","Data":"8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a"} Jul 20 20:01:10.830456 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.830130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr" event={"ID":"e81ee9d5-06e3-469a-a1e4-08d0cbc09754","Type":"ContainerDied","Data":"9cd15d209a2bfdea6c0a2f5560605305610dfe9d8da6a4fd2b76c08ff3dc9cc8"} Jul 20 20:01:10.830456 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.830153 2580 scope.go:117] "RemoveContainer" containerID="8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a" Jul 20 20:01:10.839575 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.839558 2580 scope.go:117] "RemoveContainer" containerID="90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639" Jul 20 20:01:10.847599 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.847584 2580 scope.go:117] "RemoveContainer" containerID="8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a" Jul 20 20:01:10.847831 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:01:10.847814 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a\": container with ID starting with 8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a not found: ID does not exist" containerID="8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a" Jul 20 20:01:10.847878 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.847841 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a"} err="failed to get container status \"8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a\": rpc error: code = NotFound desc = could not find container \"8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a\": container with ID starting with 8bf4953a0915bff8ca2583e27a884f03fbd1dce7db9cd2007e4113098516647a not found: ID does not exist" Jul 20 20:01:10.847878 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.847866 2580 scope.go:117] "RemoveContainer" containerID="90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639" Jul 20 20:01:10.848107 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:01:10.848089 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639\": container with ID starting with 90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639 not found: ID does not exist" containerID="90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639" Jul 20 20:01:10.848143 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.848113 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639"} err="failed to get container status \"90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639\": rpc error: code = NotFound desc = could not find container \"90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639\": container with ID starting with 90fb2936b29037472d10333c1cae04a8a044577ac4ad55bf297f5d0b298b9639 not found: ID does not exist" Jul 20 20:01:10.851492 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.851472 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr"] Jul 20 20:01:10.855824 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:10.855806 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8b313d06-openshift-default-6cf747b989-mxdbr"] Jul 20 20:01:11.061975 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:11.061945 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" path="/var/lib/kubelet/pods/e81ee9d5-06e3-469a-a1e4-08d0cbc09754/volumes" Jul 20 20:01:14.226289 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:14.226253 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:01:21.902204 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:21.902125 2580 generic.go:356] "Generic (PLEG): container finished" podID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerID="9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9" exitCode=0 Jul 20 20:01:21.902646 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:21.902198 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" event={"ID":"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905","Type":"ContainerDied","Data":"9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9"} Jul 20 20:01:21.902646 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:21.902622 2580 scope.go:117] "RemoveContainer" containerID="9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9" Jul 20 20:01:21.903629 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:21.903605 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:01:21.903707 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:21.903671 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:01:21.903707 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:21.903699 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:01:22.813920 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:22.813885 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:22.909912 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:22.909868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" event={"ID":"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905","Type":"ContainerStarted","Data":"fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179"} Jul 20 20:01:23.445627 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:23.445592 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:23.445981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:23.445958 2580 patch_prober.go:28] interesting pod/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 20 20:01:23.446139 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:23.446116 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 20 20:01:24.444913 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:24.444870 2580 patch_prober.go:28] interesting pod/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 20 20:01:24.445541 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:24.445505 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 20 20:01:25.445290 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:25.444848 2580 patch_prober.go:28] interesting pod/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 20 20:01:25.445290 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:25.444918 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 20 20:01:26.445181 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:26.445134 2580 patch_prober.go:28] interesting pod/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 20 20:01:26.445394 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:26.445221 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 20 20:01:27.447960 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:27.447925 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:27.448382 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:27.448215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:27.448826 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:27.448807 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:46.427887 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:46.427844 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:01:48.337744 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:48.337707 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7"] Jul 20 20:01:48.338516 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:48.338341 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" containerID="cri-o://fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179" gracePeriod=30 Jul 20 20:01:53.610611 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.610542 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:53.685357 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685320 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dtkz\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-kube-api-access-7dtkz\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685539 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685368 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-credential-socket\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685539 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685444 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-envoy\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685539 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685469 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-data\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685539 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685522 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-certs\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685764 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685554 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istiod-ca-cert\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685764 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685586 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-podinfo\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685764 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685629 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-token\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685764 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-socket\") pod \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\" (UID: \"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905\") " Jul 20 20:01:53.685764 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685740 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:53.686023 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.685924 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:53.686023 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.686004 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.686119 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.686022 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.686119 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.686082 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:53.686393 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.686372 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:01:53.686573 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.686548 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:53.687901 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.687856 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:01:53.688080 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.688059 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-kube-api-access-7dtkz" (OuterVolumeSpecName: "kube-api-access-7dtkz") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "kube-api-access-7dtkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:01:53.688175 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.688155 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:01:53.688175 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.688159 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" (UID: "1a2d03c2-6686-4b34-9c28-b9c5c8cb0905"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:01:53.786925 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.786885 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.786925 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.786918 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.786925 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.786930 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.787158 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.786943 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dtkz\" (UniqueName: \"kubernetes.io/projected/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-kube-api-access-7dtkz\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.787158 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.786955 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.787158 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.786966 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:53.787158 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:53.786976 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:01:54.086827 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.086794 2580 generic.go:356] "Generic (PLEG): container finished" podID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerID="fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179" exitCode=0 Jul 20 20:01:54.087011 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.086845 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" Jul 20 20:01:54.087011 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.086884 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" event={"ID":"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905","Type":"ContainerDied","Data":"fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179"} Jul 20 20:01:54.087011 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.086936 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7" event={"ID":"1a2d03c2-6686-4b34-9c28-b9c5c8cb0905","Type":"ContainerDied","Data":"ff6f621826d021ecc1e364c9b09da7a58138264505e525b1f7fd4a4c4d6496af"} Jul 20 20:01:54.087011 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.086961 2580 scope.go:117] "RemoveContainer" containerID="fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179" Jul 20 20:01:54.096884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.096867 2580 scope.go:117] "RemoveContainer" containerID="9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9" Jul 20 20:01:54.105695 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.105680 2580 scope.go:117] "RemoveContainer" containerID="fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179" Jul 20 20:01:54.105945 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:01:54.105921 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179\": container with ID starting with fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179 not found: ID does not exist" containerID="fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179" Jul 20 20:01:54.105982 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.105953 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179"} err="failed to get container status \"fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179\": rpc error: code = NotFound desc = could not find container \"fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179\": container with ID starting with fed84dac7a9a1ad3b4ee3c83d55fd9852a592803d94ac0ac800bc94cc773b179 not found: ID does not exist" Jul 20 20:01:54.105982 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.105972 2580 scope.go:117] "RemoveContainer" containerID="9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9" Jul 20 20:01:54.106182 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:01:54.106166 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9\": container with ID starting with 9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9 not found: ID does not exist" containerID="9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9" Jul 20 20:01:54.106224 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.106189 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9"} err="failed to get container status \"9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9\": rpc error: code = NotFound desc = could not find container \"9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9\": container with ID starting with 9702b1f29e04c6eaa14320ee813566ac8b8bef0765ccdcc7ce91aa78f34e71d9 not found: ID does not exist" Jul 20 20:01:54.109452 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.109407 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7"] Jul 20 20:01:54.121486 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:54.121462 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-06dcaf8a-openshift-default-f984cdc75-px2t7"] Jul 20 20:01:55.062475 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:01:55.062443 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" path="/var/lib/kubelet/pods/1a2d03c2-6686-4b34-9c28-b9c5c8cb0905/volumes" Jul 20 20:03:15.452455 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:15.450265 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:03:24.833549 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:24.833514 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:03:29.844063 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844030 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt"] Jul 20 20:03:29.844681 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844655 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" Jul 20 20:03:29.844681 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844680 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844722 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844732 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844740 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844749 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844848 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844863 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844873 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" Jul 20 20:03:29.844882 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844884 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="689d8f5f-f49d-4a12-8abf-06048a953d2d" containerName="istio-proxy" Jul 20 20:03:29.845162 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.844892 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a2d03c2-6686-4b34-9c28-b9c5c8cb0905" containerName="istio-proxy" Jul 20 20:03:29.848378 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.848359 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.850504 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.850479 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-166791b0-openshift-default-dockercfg-s6tfm\"" Jul 20 20:03:29.875155 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.875127 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt"] Jul 20 20:03:29.978412 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-data\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978412 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978418 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-credential-socket\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978658 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978470 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57235de1-87aa-44ef-a55f-b4502beb2794-istiod-ca-cert\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978658 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978487 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-certs\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978658 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-istio-token\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978658 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-socket\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978658 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57235de1-87aa-44ef-a55f-b4502beb2794-istio-podinfo\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978836 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-envoy\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:29.978836 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:29.978700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htdd4\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-kube-api-access-htdd4\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.079910 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.079875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-data\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080077 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.079917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-credential-socket\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080126 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57235de1-87aa-44ef-a55f-b4502beb2794-istiod-ca-cert\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080162 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-certs\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080162 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080154 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-istio-token\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080261 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080244 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-socket\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080304 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57235de1-87aa-44ef-a55f-b4502beb2794-istio-podinfo\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080304 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-data\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080372 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-envoy\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080372 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080340 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-credential-socket\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080372 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdd4\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-kube-api-access-htdd4\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080534 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080512 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-certs\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080600 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-socket\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.080805 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.080784 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57235de1-87aa-44ef-a55f-b4502beb2794-istiod-ca-cert\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.082629 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.082603 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-envoy\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.082877 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.082857 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57235de1-87aa-44ef-a55f-b4502beb2794-istio-podinfo\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.091942 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.091914 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-istio-token\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.092014 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.091964 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htdd4\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-kube-api-access-htdd4\") pod \"e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.159446 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.159341 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:30.313451 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.313406 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt"] Jul 20 20:03:30.314488 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:03:30.314458 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57235de1_87aa_44ef_a55f_b4502beb2794.slice/crio-9b5224e1ba332e002971e9fd77cfca320000311fddf956677ca1dae7305bca16 WatchSource:0}: Error finding container 9b5224e1ba332e002971e9fd77cfca320000311fddf956677ca1dae7305bca16: Status 404 returned error can't find the container with id 9b5224e1ba332e002971e9fd77cfca320000311fddf956677ca1dae7305bca16 Jul 20 20:03:30.316686 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.316657 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:03:30.316790 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.316721 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:03:30.316790 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.316752 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:03:30.599666 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.599633 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" event={"ID":"57235de1-87aa-44ef-a55f-b4502beb2794","Type":"ContainerStarted","Data":"910f2d97ec982affc8361f774ec6edeb5b588e8450012e93a75cac160249512c"} Jul 20 20:03:30.599881 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:30.599685 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" event={"ID":"57235de1-87aa-44ef-a55f-b4502beb2794","Type":"ContainerStarted","Data":"9b5224e1ba332e002971e9fd77cfca320000311fddf956677ca1dae7305bca16"} Jul 20 20:03:31.159861 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:31.159817 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:31.164925 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:31.164899 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:31.187236 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:31.187184 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" podStartSLOduration=2.187167277 podStartE2EDuration="2.187167277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:03:30.660092544 +0000 UTC m=+2152.285171539" watchObservedRunningTime="2026-07-20 20:03:31.187167277 +0000 UTC m=+2152.812246271" Jul 20 20:03:31.605572 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:31.605532 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:31.606543 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:31.606522 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:35.740300 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:35.740265 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:03:44.689202 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:44.689165 2580 generic.go:356] "Generic (PLEG): container finished" podID="57235de1-87aa-44ef-a55f-b4502beb2794" containerID="910f2d97ec982affc8361f774ec6edeb5b588e8450012e93a75cac160249512c" exitCode=0 Jul 20 20:03:44.689657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:44.689229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" event={"ID":"57235de1-87aa-44ef-a55f-b4502beb2794","Type":"ContainerDied","Data":"910f2d97ec982affc8361f774ec6edeb5b588e8450012e93a75cac160249512c"} Jul 20 20:03:44.689729 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:44.689711 2580 scope.go:117] "RemoveContainer" containerID="910f2d97ec982affc8361f774ec6edeb5b588e8450012e93a75cac160249512c" Jul 20 20:03:44.690813 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:44.690788 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:03:44.690879 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:44.690866 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:03:44.690917 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:44.690900 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:03:45.697194 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:45.697161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" event={"ID":"57235de1-87aa-44ef-a55f-b4502beb2794","Type":"ContainerStarted","Data":"bbfeb71569c3edddd361aa58ba15b50d0e1588e0af238302ca71bacbc643a2b3"} Jul 20 20:03:46.159906 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:46.159863 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:46.161078 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:46.161050 2580 patch_prober.go:28] interesting pod/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 20 20:03:46.161178 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:46.161115 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 20 20:03:46.606032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:46.605979 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:47.160769 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:47.160729 2580 patch_prober.go:28] interesting pod/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 20 20:03:47.161209 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:47.160817 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 20 20:03:48.159987 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:48.159943 2580 patch_prober.go:28] interesting pod/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 20 20:03:48.160171 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:48.160013 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 20 20:03:49.159763 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:49.159720 2580 patch_prober.go:28] interesting pod/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 20 20:03:49.160226 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:49.159790 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 20 20:03:50.163695 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:50.163661 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:50.164638 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:50.164618 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:03:58.538003 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:03:58.537965 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:04:06.652231 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:06.652191 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt"] Jul 20 20:04:06.652705 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:06.652414 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" containerID="cri-o://bbfeb71569c3edddd361aa58ba15b50d0e1588e0af238302ca71bacbc643a2b3" gracePeriod=30 Jul 20 20:04:11.844901 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.844850 2580 generic.go:356] "Generic (PLEG): container finished" podID="57235de1-87aa-44ef-a55f-b4502beb2794" containerID="bbfeb71569c3edddd361aa58ba15b50d0e1588e0af238302ca71bacbc643a2b3" exitCode=0 Jul 20 20:04:11.845359 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.844899 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" event={"ID":"57235de1-87aa-44ef-a55f-b4502beb2794","Type":"ContainerDied","Data":"bbfeb71569c3edddd361aa58ba15b50d0e1588e0af238302ca71bacbc643a2b3"} Jul 20 20:04:11.845359 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.845000 2580 scope.go:117] "RemoveContainer" containerID="910f2d97ec982affc8361f774ec6edeb5b588e8450012e93a75cac160249512c" Jul 20 20:04:11.912306 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.912243 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:04:11.979222 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979176 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-socket\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979423 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979250 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-credential-socket\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979423 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979305 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-certs\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979423 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979324 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htdd4\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-kube-api-access-htdd4\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979423 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979352 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57235de1-87aa-44ef-a55f-b4502beb2794-istiod-ca-cert\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979423 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979386 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-data\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979423 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979406 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-envoy\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979776 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979483 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57235de1-87aa-44ef-a55f-b4502beb2794-istio-podinfo\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.979776 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979522 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:04:11.979776 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979670 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:04:11.979776 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979662 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:04:11.979776 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979740 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-istio-token\") pod \"57235de1-87aa-44ef-a55f-b4502beb2794\" (UID: \"57235de1-87aa-44ef-a55f-b4502beb2794\") " Jul 20 20:04:11.980050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.979848 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57235de1-87aa-44ef-a55f-b4502beb2794-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:04:11.980132 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.980113 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:11.980194 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.980137 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:11.980194 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.980150 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57235de1-87aa-44ef-a55f-b4502beb2794-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:11.980194 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.980160 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:11.980358 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.980328 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-data" (OuterVolumeSpecName: "istio-data") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:04:11.981779 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.981751 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57235de1-87aa-44ef-a55f-b4502beb2794-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:04:11.981897 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.981795 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:04:11.982043 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.982021 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-kube-api-access-htdd4" (OuterVolumeSpecName: "kube-api-access-htdd4") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "kube-api-access-htdd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:04:11.982043 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:11.982028 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-istio-token" (OuterVolumeSpecName: "istio-token") pod "57235de1-87aa-44ef-a55f-b4502beb2794" (UID: "57235de1-87aa-44ef-a55f-b4502beb2794"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:04:12.081373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.081335 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htdd4\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-kube-api-access-htdd4\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:12.081373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.081368 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:12.081373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.081377 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57235de1-87aa-44ef-a55f-b4502beb2794-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:12.081373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.081384 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57235de1-87aa-44ef-a55f-b4502beb2794-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:12.081673 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.081392 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57235de1-87aa-44ef-a55f-b4502beb2794-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:04:12.851740 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.851714 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" Jul 20 20:04:12.852225 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.851714 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt" event={"ID":"57235de1-87aa-44ef-a55f-b4502beb2794","Type":"ContainerDied","Data":"9b5224e1ba332e002971e9fd77cfca320000311fddf956677ca1dae7305bca16"} Jul 20 20:04:12.852225 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.851855 2580 scope.go:117] "RemoveContainer" containerID="bbfeb71569c3edddd361aa58ba15b50d0e1588e0af238302ca71bacbc643a2b3" Jul 20 20:04:12.875270 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.875235 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt"] Jul 20 20:04:12.878890 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:12.878864 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-166791b0-openshift-default-5cf7c8579-jlsnt"] Jul 20 20:04:13.062979 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:13.062947 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" path="/var/lib/kubelet/pods/57235de1-87aa-44ef-a55f-b4502beb2794/volumes" Jul 20 20:04:26.836228 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836180 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt"] Jul 20 20:04:26.836728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836630 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" Jul 20 20:04:26.836728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836649 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" Jul 20 20:04:26.836728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836674 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" Jul 20 20:04:26.836728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836680 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" Jul 20 20:04:26.836728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836688 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" Jul 20 20:04:26.836728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836694 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e81ee9d5-06e3-469a-a1e4-08d0cbc09754" containerName="istio-proxy" Jul 20 20:04:26.836944 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.836767 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" Jul 20 20:04:26.838834 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.838814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.840845 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.840824 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0243e2c4-openshift-default-dockercfg-xfm57\"" Jul 20 20:04:26.851697 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.851673 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt"] Jul 20 20:04:26.929233 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929193 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-certs\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929454 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdgdt\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-kube-api-access-xdgdt\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929454 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-credential-socket\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929454 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-token\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929454 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-podinfo\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929454 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-socket\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929454 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-envoy\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929753 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929509 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-data\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:26.929753 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:26.929553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcfeecab-643f-4a62-9ad8-193684a9d123-istiod-ca-cert\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.030674 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.030625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcfeecab-643f-4a62-9ad8-193684a9d123-istiod-ca-cert\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.030868 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.030693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-certs\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.030868 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.030749 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdgdt\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-kube-api-access-xdgdt\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.030868 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.030810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-credential-socket\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.030868 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.030855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-token\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031110 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.030888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-podinfo\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031110 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.030987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-socket\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031110 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.031054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-envoy\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031261 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.031132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-data\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031261 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.031186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-credential-socket\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031375 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.031350 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-certs\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031502 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.031474 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcfeecab-643f-4a62-9ad8-193684a9d123-istiod-ca-cert\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031502 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.031496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-data\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.031706 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.031524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-socket\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.033579 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.033559 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-envoy\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.033791 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.033771 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-podinfo\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.037976 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.037950 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-token\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.038123 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.038105 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdgdt\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-kube-api-access-xdgdt\") pod \"e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.150179 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.150080 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:27.292810 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.292782 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt"] Jul 20 20:04:27.294688 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:04:27.294649 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfeecab_643f_4a62_9ad8_193684a9d123.slice/crio-f9a6afac07b52adf47371daf03dde50fd530737a899df5fe2f5e1b7a2aaa1de2 WatchSource:0}: Error finding container f9a6afac07b52adf47371daf03dde50fd530737a899df5fe2f5e1b7a2aaa1de2: Status 404 returned error can't find the container with id f9a6afac07b52adf47371daf03dde50fd530737a899df5fe2f5e1b7a2aaa1de2 Jul 20 20:04:27.296916 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.296884 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:04:27.297006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.296966 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:04:27.297006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.296995 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:04:27.933609 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.933565 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" event={"ID":"bcfeecab-643f-4a62-9ad8-193684a9d123","Type":"ContainerStarted","Data":"a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036"} Jul 20 20:04:27.934081 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.933666 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" event={"ID":"bcfeecab-643f-4a62-9ad8-193684a9d123","Type":"ContainerStarted","Data":"f9a6afac07b52adf47371daf03dde50fd530737a899df5fe2f5e1b7a2aaa1de2"} Jul 20 20:04:27.955240 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:27.955183 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" podStartSLOduration=1.955166985 podStartE2EDuration="1.955166985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:04:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:04:27.952723262 +0000 UTC m=+2209.577802267" watchObservedRunningTime="2026-07-20 20:04:27.955166985 +0000 UTC m=+2209.580245979" Jul 20 20:04:28.151372 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:28.151330 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:28.156194 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:28.156163 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:28.938849 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:28.938816 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:28.939765 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:28.939743 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:36.327682 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:36.327640 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:04:44.069860 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:44.069657 2580 patch_prober.go:28] interesting pod/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:34178->10.133.0.54:15021: read: connection reset by peer" start-of-body= Jul 20 20:04:44.069860 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:44.069727 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:34178->10.133.0.54:15021: read: connection reset by peer" Jul 20 20:04:45.028919 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:45.028882 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerID="a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036" exitCode=0 Jul 20 20:04:45.028919 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:45.028921 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" event={"ID":"bcfeecab-643f-4a62-9ad8-193684a9d123","Type":"ContainerDied","Data":"a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036"} Jul 20 20:04:45.029305 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:45.029291 2580 scope.go:117] "RemoveContainer" containerID="a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036" Jul 20 20:04:45.030856 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:45.030811 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:04:45.031146 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:45.031123 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:04:45.031343 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:45.031317 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:04:46.040998 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:46.040962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" event={"ID":"bcfeecab-643f-4a62-9ad8-193684a9d123","Type":"ContainerStarted","Data":"a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363"} Jul 20 20:04:46.151015 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:46.150979 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:46.151263 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:46.151235 2580 patch_prober.go:28] interesting pod/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 20 20:04:46.151332 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:46.151289 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 20 20:04:47.150917 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:47.150876 2580 patch_prober.go:28] interesting pod/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 20 20:04:47.151341 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:47.150933 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 20 20:04:48.151490 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:48.151446 2580 patch_prober.go:28] interesting pod/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 20 20:04:48.152027 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:48.151522 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 20 20:04:49.152101 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:49.152052 2580 patch_prober.go:28] interesting pod/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 20 20:04:49.152543 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:49.152127 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 20 20:04:50.154445 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:50.154397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:50.156224 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:50.156198 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:50.157593 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:50.157565 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:04:51.434313 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:04:51.434250 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:05:06.623762 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:05:06.623678 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:05:27.674290 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:05:27.674249 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:05:38.242782 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:05:38.242721 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:06:25.583052 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:25.583017 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt"] Jul 20 20:06:25.583594 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:25.583265 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" containerID="cri-o://a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363" gracePeriod=30 Jul 20 20:06:30.834783 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.834754 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:06:30.840412 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840387 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-data\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840556 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-token\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840628 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840591 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-credential-socket\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840628 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840624 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-certs\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840737 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840654 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-podinfo\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840795 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840733 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdgdt\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-kube-api-access-xdgdt\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840795 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840767 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcfeecab-643f-4a62-9ad8-193684a9d123-istiod-ca-cert\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840937 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840898 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:06:30.840937 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840901 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-envoy\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.840937 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.840922 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:06:30.841109 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.841089 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcfeecab-643f-4a62-9ad8-193684a9d123-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:06:30.841249 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.841208 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-data" (OuterVolumeSpecName: "istio-data") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:06:30.841373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.841282 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcfeecab-643f-4a62-9ad8-193684a9d123-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:30.841373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.841300 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:30.841373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.841312 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:30.841373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.841328 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:30.842834 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.842809 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-token" (OuterVolumeSpecName: "istio-token") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:06:30.842937 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.842911 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:06:30.843487 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.843462 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:06:30.843556 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.843476 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-kube-api-access-xdgdt" (OuterVolumeSpecName: "kube-api-access-xdgdt") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "kube-api-access-xdgdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:06:30.941620 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.941588 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-socket\") pod \"bcfeecab-643f-4a62-9ad8-193684a9d123\" (UID: \"bcfeecab-643f-4a62-9ad8-193684a9d123\") " Jul 20 20:06:30.941826 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.941816 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:30.941884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.941829 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:30.941884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.941824 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bcfeecab-643f-4a62-9ad8-193684a9d123" (UID: "bcfeecab-643f-4a62-9ad8-193684a9d123"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:06:30.941884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.941839 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdgdt\" (UniqueName: \"kubernetes.io/projected/bcfeecab-643f-4a62-9ad8-193684a9d123-kube-api-access-xdgdt\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:30.941884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:30.941849 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:31.043175 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.043139 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcfeecab-643f-4a62-9ad8-193684a9d123-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:06:31.611001 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.610969 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerID="a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363" exitCode=0 Jul 20 20:06:31.611189 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.611017 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" Jul 20 20:06:31.611189 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.611062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" event={"ID":"bcfeecab-643f-4a62-9ad8-193684a9d123","Type":"ContainerDied","Data":"a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363"} Jul 20 20:06:31.611189 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.611117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt" event={"ID":"bcfeecab-643f-4a62-9ad8-193684a9d123","Type":"ContainerDied","Data":"f9a6afac07b52adf47371daf03dde50fd530737a899df5fe2f5e1b7a2aaa1de2"} Jul 20 20:06:31.611189 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.611140 2580 scope.go:117] "RemoveContainer" containerID="a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363" Jul 20 20:06:31.620308 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.620291 2580 scope.go:117] "RemoveContainer" containerID="a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036" Jul 20 20:06:31.628876 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.628852 2580 scope.go:117] "RemoveContainer" containerID="a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363" Jul 20 20:06:31.629232 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:06:31.629214 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363\": container with ID starting with a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363 not found: ID does not exist" containerID="a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363" Jul 20 20:06:31.629280 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.629243 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363"} err="failed to get container status \"a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363\": rpc error: code = NotFound desc = could not find container \"a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363\": container with ID starting with a8bd0d921cd22387ecab015c373bad64916cb52cbdbcd17d4220dab44d6cc363 not found: ID does not exist" Jul 20 20:06:31.629280 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.629261 2580 scope.go:117] "RemoveContainer" containerID="a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036" Jul 20 20:06:31.629519 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:06:31.629502 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036\": container with ID starting with a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036 not found: ID does not exist" containerID="a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036" Jul 20 20:06:31.629579 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.629523 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036"} err="failed to get container status \"a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036\": rpc error: code = NotFound desc = could not find container \"a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036\": container with ID starting with a601a102401e0e50338ab106c6056a2328121fa13f2989d6134c25bb533df036 not found: ID does not exist" Jul 20 20:06:31.630222 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.630204 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt"] Jul 20 20:06:31.636200 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:31.636176 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0243e2c4-openshift-default-7c95c475cf-j5bjt"] Jul 20 20:06:33.062392 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:33.062356 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" path="/var/lib/kubelet/pods/bcfeecab-643f-4a62-9ad8-193684a9d123/volumes" Jul 20 20:06:35.839332 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:35.839297 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:06:43.238124 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:06:43.238085 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:07:22.935792 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.935754 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664"] Jul 20 20:07:22.936316 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.936154 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" Jul 20 20:07:22.936316 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.936166 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" Jul 20 20:07:22.936316 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.936175 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" Jul 20 20:07:22.936316 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.936181 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" Jul 20 20:07:22.936316 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.936233 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" Jul 20 20:07:22.936316 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.936243 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57235de1-87aa-44ef-a55f-b4502beb2794" containerName="istio-proxy" Jul 20 20:07:22.939267 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.939248 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:22.943320 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.943295 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b42b86-openshift-default-dockercfg-npt7q\"" Jul 20 20:07:22.975226 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:22.975190 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664"] Jul 20 20:07:23.023131 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istiod-ca-cert\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023334 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-podinfo\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023334 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89wfn\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-kube-api-access-89wfn\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023334 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-token\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023585 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023367 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-certs\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023585 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-data\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023585 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023403 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-socket\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023585 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023477 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-envoy\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.023585 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.023520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-credential-socket\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124703 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istiod-ca-cert\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-podinfo\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89wfn\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-kube-api-access-89wfn\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-token\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-certs\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-data\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-socket\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.124902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.124891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-envoy\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.125292 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.125218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-credential-socket\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.125389 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.125363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-socket\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.125621 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.125560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istiod-ca-cert\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.125621 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.125607 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-credential-socket\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.125837 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.125648 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-data\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.125837 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.125686 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-certs\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.127362 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.127338 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-envoy\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.127515 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.127496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-podinfo\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.155035 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.155001 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-token\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.158166 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.158140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89wfn\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-kube-api-access-89wfn\") pod \"e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.250695 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.250603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:23.404476 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.404446 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664"] Jul 20 20:07:23.405172 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:07:23.405147 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad44a5b4_c2b8_4a80_add4_5c72e9dcb3a0.slice/crio-cb2e605625231c617b211f745fe4d304bae12ed1637fdc62e1a8847300eddef2 WatchSource:0}: Error finding container cb2e605625231c617b211f745fe4d304bae12ed1637fdc62e1a8847300eddef2: Status 404 returned error can't find the container with id cb2e605625231c617b211f745fe4d304bae12ed1637fdc62e1a8847300eddef2 Jul 20 20:07:23.407293 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.407260 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:23.407370 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.407330 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:23.407370 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.407363 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:23.893528 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.893493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" event={"ID":"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0","Type":"ContainerStarted","Data":"9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524"} Jul 20 20:07:23.893749 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.893551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" event={"ID":"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0","Type":"ContainerStarted","Data":"cb2e605625231c617b211f745fe4d304bae12ed1637fdc62e1a8847300eddef2"} Jul 20 20:07:23.948902 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:23.948850 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" podStartSLOduration=1.948833665 podStartE2EDuration="1.948833665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:07:23.946911653 +0000 UTC m=+2385.571990646" watchObservedRunningTime="2026-07-20 20:07:23.948833665 +0000 UTC m=+2385.573912637" Jul 20 20:07:24.251387 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:24.251299 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:24.256006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:24.255981 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:24.900234 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:24.900202 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:24.901027 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:24.901004 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:31.333601 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.333563 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl"] Jul 20 20:07:31.334375 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.334353 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcfeecab-643f-4a62-9ad8-193684a9d123" containerName="istio-proxy" Jul 20 20:07:31.338112 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.338085 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.340371 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.340351 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-850607-openshift-default-dockercfg-7ssp8\"" Jul 20 20:07:31.351561 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.351541 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl"] Jul 20 20:07:31.411360 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns2r9\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-kube-api-access-ns2r9\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-podinfo\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-data\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-envoy\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411543 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-credential-socket\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-certs\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-token\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411822 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istiod-ca-cert\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.411822 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.411619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-socket\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512335 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512293 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-data\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512335 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-envoy\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512466 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-credential-socket\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-certs\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-token\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istiod-ca-cert\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-socket\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns2r9\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-kube-api-access-ns2r9\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512965 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-podinfo\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512965 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512835 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-data\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.512965 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512926 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-certs\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.513127 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.512969 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-socket\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.513187 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.513172 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-credential-socket\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.513305 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.513285 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istiod-ca-cert\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.515138 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.515118 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-envoy\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.515349 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.515332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-podinfo\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.522955 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.522927 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-token\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.522955 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.522948 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns2r9\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-kube-api-access-ns2r9\") pod \"e2e-inf-b-850607-openshift-default-74db579b75-47sjl\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.649768 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.649668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:31.657140 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.657100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:07:31.806242 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.806212 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl"] Jul 20 20:07:31.806671 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:07:31.806643 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod685c9f78_02fb_4ead_b9a5_d96d2d3a65c0.slice/crio-84841e7141e13c8d2c64a27bc9d128d8143a56f003746c44022e9d7f53aec849 WatchSource:0}: Error finding container 84841e7141e13c8d2c64a27bc9d128d8143a56f003746c44022e9d7f53aec849: Status 404 returned error can't find the container with id 84841e7141e13c8d2c64a27bc9d128d8143a56f003746c44022e9d7f53aec849 Jul 20 20:07:31.808747 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.808718 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:31.808835 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.808777 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:31.808835 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.808808 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:31.940898 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.940801 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" event={"ID":"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0","Type":"ContainerStarted","Data":"df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9"} Jul 20 20:07:31.940898 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.940862 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" event={"ID":"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0","Type":"ContainerStarted","Data":"84841e7141e13c8d2c64a27bc9d128d8143a56f003746c44022e9d7f53aec849"} Jul 20 20:07:31.960308 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:31.960258 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" podStartSLOduration=0.960239538 podStartE2EDuration="960.239538ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:07:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:07:31.959112605 +0000 UTC m=+2393.584191610" watchObservedRunningTime="2026-07-20 20:07:31.960239538 +0000 UTC m=+2393.585318532" Jul 20 20:07:32.650751 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:32.650696 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:32.656965 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:32.656932 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:32.950708 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:32.950610 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:32.951764 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:32.951735 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:39.901304 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:39.901221 2580 patch_prober.go:28] interesting pod/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 20:07:39.901304 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:39.901274 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 20:07:40.000244 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:40.000153 2580 generic.go:356] "Generic (PLEG): container finished" podID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerID="9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524" exitCode=0 Jul 20 20:07:40.000244 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:40.000228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" event={"ID":"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0","Type":"ContainerDied","Data":"9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524"} Jul 20 20:07:40.000710 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:40.000693 2580 scope.go:117] "RemoveContainer" containerID="9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524" Jul 20 20:07:40.001733 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:40.001704 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:40.001801 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:40.001780 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:40.001839 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:40.001810 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:40.224399 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:40.224359 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:07:41.014910 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:41.014874 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" event={"ID":"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0","Type":"ContainerStarted","Data":"4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08"} Jul 20 20:07:41.251483 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:41.251446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:41.251941 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:41.251906 2580 patch_prober.go:28] interesting pod/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 20:07:41.252175 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:41.252131 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 20:07:42.251390 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:42.251349 2580 patch_prober.go:28] interesting pod/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 20:07:42.252080 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:42.252043 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 20:07:43.251243 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:43.251191 2580 patch_prober.go:28] interesting pod/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 20:07:43.251489 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:43.251266 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 20:07:44.252311 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:44.251830 2580 patch_prober.go:28] interesting pod/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 20 20:07:44.252311 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:44.251920 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 20 20:07:45.258372 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:45.258338 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:45.258794 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:45.258737 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:45.259498 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:45.259477 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:07:48.129113 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:48.128378 2580 patch_prober.go:28] interesting pod/e2e-inf-b-850607-openshift-default-74db579b75-47sjl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": read tcp 10.133.0.2:59230->10.133.0.56:15021: read: connection reset by peer" start-of-body= Jul 20 20:07:48.129113 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:48.128503 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": read tcp 10.133.0.2:59230->10.133.0.56:15021: read: connection reset by peer" Jul 20 20:07:49.070870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:49.070838 2580 generic.go:356] "Generic (PLEG): container finished" podID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerID="df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9" exitCode=0 Jul 20 20:07:49.071079 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:49.070911 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" event={"ID":"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0","Type":"ContainerDied","Data":"df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9"} Jul 20 20:07:49.071307 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:49.071291 2580 scope.go:117] "RemoveContainer" containerID="df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9" Jul 20 20:07:49.072284 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:49.072260 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:49.072341 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:49.072330 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:49.072377 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:49.072364 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:07:50.078508 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:50.078469 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" event={"ID":"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0","Type":"ContainerStarted","Data":"e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75"} Jul 20 20:07:50.650235 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:50.650188 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:50.650620 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:50.650588 2580 patch_prober.go:28] interesting pod/e2e-inf-b-850607-openshift-default-74db579b75-47sjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 20:07:50.650795 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:50.650651 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 20:07:51.650098 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:51.650053 2580 patch_prober.go:28] interesting pod/e2e-inf-b-850607-openshift-default-74db579b75-47sjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 20:07:51.650526 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:51.650117 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 20:07:52.650559 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:52.650516 2580 patch_prober.go:28] interesting pod/e2e-inf-b-850607-openshift-default-74db579b75-47sjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 20:07:52.650981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:52.650587 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 20:07:53.650487 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:53.650443 2580 patch_prober.go:28] interesting pod/e2e-inf-b-850607-openshift-default-74db579b75-47sjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 20 20:07:53.650713 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:53.650520 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 20 20:07:54.653849 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:54.653810 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:54.654243 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:54.654131 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:07:54.654980 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:07:54.654963 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:08:05.950339 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:08:05.950250 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:08:16.828274 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:08:16.828230 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:08:31.531663 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:08:31.531623 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:08:45.328350 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:08:45.328312 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:09:18.636772 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:18.636731 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:09:33.641671 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:33.641577 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:09:36.825938 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:36.825899 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664"] Jul 20 20:09:36.826498 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:36.826141 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" containerID="cri-o://4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08" gracePeriod=30 Jul 20 20:09:42.085644 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.085576 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:09:42.220133 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220095 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istiod-ca-cert\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220336 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220162 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-envoy\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220336 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220275 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-token\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220336 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220326 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-credential-socket\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220540 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220353 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-socket\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220540 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220472 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-certs\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220540 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220510 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89wfn\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-kube-api-access-89wfn\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220540 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220527 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:09:42.220750 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220536 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-data\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220750 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220731 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-podinfo\") pod \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\" (UID: \"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0\") " Jul 20 20:09:42.220859 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220637 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:42.220859 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220660 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:42.220859 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.220765 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:42.221120 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.221099 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.221174 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.221126 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.221174 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.221144 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.221174 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.221158 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.221358 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.221336 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:42.222714 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.222686 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-kube-api-access-89wfn" (OuterVolumeSpecName: "kube-api-access-89wfn") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "kube-api-access-89wfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:09:42.222837 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.222753 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:09:42.222837 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.222821 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:42.222920 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.222850 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" (UID: "ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:09:42.321664 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.321631 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.321664 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.321661 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89wfn\" (UniqueName: \"kubernetes.io/projected/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-kube-api-access-89wfn\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.321664 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.321670 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.321942 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.321679 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.321942 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.321687 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:09:42.694835 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.694795 2580 generic.go:356] "Generic (PLEG): container finished" podID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerID="4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08" exitCode=0 Jul 20 20:09:42.695045 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.694861 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" Jul 20 20:09:42.695045 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.694873 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" event={"ID":"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0","Type":"ContainerDied","Data":"4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08"} Jul 20 20:09:42.695045 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.694949 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664" event={"ID":"ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0","Type":"ContainerDied","Data":"cb2e605625231c617b211f745fe4d304bae12ed1637fdc62e1a8847300eddef2"} Jul 20 20:09:42.695045 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.694973 2580 scope.go:117] "RemoveContainer" containerID="4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08" Jul 20 20:09:42.705085 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.705068 2580 scope.go:117] "RemoveContainer" containerID="9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524" Jul 20 20:09:42.713619 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.713600 2580 scope.go:117] "RemoveContainer" containerID="4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08" Jul 20 20:09:42.713900 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:09:42.713882 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08\": container with ID starting with 4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08 not found: ID does not exist" containerID="4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08" Jul 20 20:09:42.713963 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.713910 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08"} err="failed to get container status \"4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08\": rpc error: code = NotFound desc = could not find container \"4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08\": container with ID starting with 4fb18d028e8f0b4633fbf8daa2733614fa508479a9025740b12a08a19e6a1e08 not found: ID does not exist" Jul 20 20:09:42.713963 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.713929 2580 scope.go:117] "RemoveContainer" containerID="9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524" Jul 20 20:09:42.714200 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:09:42.714180 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524\": container with ID starting with 9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524 not found: ID does not exist" containerID="9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524" Jul 20 20:09:42.714268 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.714207 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524"} err="failed to get container status \"9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524\": rpc error: code = NotFound desc = could not find container \"9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524\": container with ID starting with 9a439a7ac84cb3d7da95de1d96b48f0ad5d67d09c4fdad0f7572f0ccc3144524 not found: ID does not exist" Jul 20 20:09:42.719224 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.719198 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664"] Jul 20 20:09:42.723214 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:42.723190 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b42b86-openshift-default-5dddd8f5d5-hr664"] Jul 20 20:09:43.062722 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:43.062692 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" path="/var/lib/kubelet/pods/ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0/volumes" Jul 20 20:09:50.337742 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:50.337701 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:09:58.848479 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:58.848418 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl"] Jul 20 20:09:58.848892 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:58.848659 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" containerID="cri-o://e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75" gracePeriod=30 Jul 20 20:09:59.871085 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.871050 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw"] Jul 20 20:09:59.871511 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.871439 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" Jul 20 20:09:59.871511 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.871451 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" Jul 20 20:09:59.871511 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.871472 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" Jul 20 20:09:59.871511 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.871477 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" Jul 20 20:09:59.871671 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.871537 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" Jul 20 20:09:59.871671 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.871546 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad44a5b4-c2b8-4a80-add4-5c72e9dcb3a0" containerName="istio-proxy" Jul 20 20:09:59.874694 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.874667 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.876927 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.876901 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b18634-openshift-default-dockercfg-zqnx7\"" Jul 20 20:09:59.887833 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.887800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw"] Jul 20 20:09:59.991571 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-socket\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.991762 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-podinfo\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.991762 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-certs\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.991762 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-data\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.991762 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991667 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-token\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.991762 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991743 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-envoy\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.992039 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991830 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-credential-socket\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.992039 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzslk\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-kube-api-access-wzslk\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:09:59.992039 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:09:59.991874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istiod-ca-cert\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092791 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-credential-socket\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzslk\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-kube-api-access-wzslk\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istiod-ca-cert\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-socket\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-podinfo\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-certs\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092922 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-data\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.092981 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.092970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-token\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.093365 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.093016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-envoy\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.093365 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.093256 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-credential-socket\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.093500 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.093378 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-certs\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.093500 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.093413 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-socket\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.093750 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.093682 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istiod-ca-cert\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.093851 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.093776 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-data\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.095470 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.095414 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-envoy\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.095570 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.095484 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-podinfo\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.100373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.100342 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-token\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.100496 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.100415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzslk\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-kube-api-access-wzslk\") pod \"e2e-api-a-b18634-openshift-default-96f66c867-trbmw\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.186915 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.186816 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:00.340787 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.340645 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw"] Jul 20 20:10:00.343079 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:10:00.343040 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fda4f6c_c6b2_4d22_82d6_1d8877329454.slice/crio-1e5f7909411f0d8f5281645443617c6245fd39d7b20a3baa430e4089563928b3 WatchSource:0}: Error finding container 1e5f7909411f0d8f5281645443617c6245fd39d7b20a3baa430e4089563928b3: Status 404 returned error can't find the container with id 1e5f7909411f0d8f5281645443617c6245fd39d7b20a3baa430e4089563928b3 Jul 20 20:10:00.345147 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.345115 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:00.345224 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.345182 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:00.345224 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.345208 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:00.798203 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.798164 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" event={"ID":"7fda4f6c-c6b2-4d22-82d6-1d8877329454","Type":"ContainerStarted","Data":"1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff"} Jul 20 20:10:00.798360 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.798218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" event={"ID":"7fda4f6c-c6b2-4d22-82d6-1d8877329454","Type":"ContainerStarted","Data":"1e5f7909411f0d8f5281645443617c6245fd39d7b20a3baa430e4089563928b3"} Jul 20 20:10:00.819795 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:00.819741 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podStartSLOduration=1.819720067 podStartE2EDuration="1.819720067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:10:00.819050613 +0000 UTC m=+2542.444129609" watchObservedRunningTime="2026-07-20 20:10:00.819720067 +0000 UTC m=+2542.444799062" Jul 20 20:10:01.187199 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:01.187099 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:01.192362 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:01.192331 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:01.804567 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:01.804533 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:01.805490 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:01.805467 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:04.096095 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.096067 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:10:04.231208 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231124 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istiod-ca-cert\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231208 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231167 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns2r9\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-kube-api-access-ns2r9\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231208 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231189 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-certs\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231528 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231233 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-data\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231528 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231259 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-token\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231528 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231308 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-credential-socket\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231528 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231344 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-podinfo\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231528 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231488 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:04.231790 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231666 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-envoy\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231790 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231691 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:04.231790 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231726 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-socket\") pod \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\" (UID: \"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0\") " Jul 20 20:10:04.231953 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.231836 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:10:04.232170 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.232149 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.232232 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.232176 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.232232 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.232177 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:04.232232 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.232191 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.232341 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.232226 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:04.233703 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.233671 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:04.233814 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.233728 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:10:04.233905 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.233882 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:10:04.234007 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.233955 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-kube-api-access-ns2r9" (OuterVolumeSpecName: "kube-api-access-ns2r9") pod "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" (UID: "685c9f78-02fb-4ead-b9a5-d96d2d3a65c0"). InnerVolumeSpecName "kube-api-access-ns2r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:10:04.332901 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.332865 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.332901 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.332899 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.332901 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.332909 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns2r9\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-kube-api-access-ns2r9\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.333154 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.332917 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.333154 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.332927 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.333154 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.332935 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:04.823276 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.823239 2580 generic.go:356] "Generic (PLEG): container finished" podID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerID="e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75" exitCode=0 Jul 20 20:10:04.823526 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.823289 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" Jul 20 20:10:04.823526 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.823322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" event={"ID":"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0","Type":"ContainerDied","Data":"e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75"} Jul 20 20:10:04.823526 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.823381 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl" event={"ID":"685c9f78-02fb-4ead-b9a5-d96d2d3a65c0","Type":"ContainerDied","Data":"84841e7141e13c8d2c64a27bc9d128d8143a56f003746c44022e9d7f53aec849"} Jul 20 20:10:04.823526 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.823404 2580 scope.go:117] "RemoveContainer" containerID="e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75" Jul 20 20:10:04.833247 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.833223 2580 scope.go:117] "RemoveContainer" containerID="df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9" Jul 20 20:10:04.842300 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.842273 2580 scope.go:117] "RemoveContainer" containerID="e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75" Jul 20 20:10:04.842618 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:10:04.842598 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75\": container with ID starting with e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75 not found: ID does not exist" containerID="e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75" Jul 20 20:10:04.842687 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.842630 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75"} err="failed to get container status \"e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75\": rpc error: code = NotFound desc = could not find container \"e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75\": container with ID starting with e27ee3b037639d5c258df0fdb37e07d73235adfb074ba02e46361464516e4a75 not found: ID does not exist" Jul 20 20:10:04.842687 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.842654 2580 scope.go:117] "RemoveContainer" containerID="df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9" Jul 20 20:10:04.842882 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:10:04.842866 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9\": container with ID starting with df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9 not found: ID does not exist" containerID="df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9" Jul 20 20:10:04.842923 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.842885 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9"} err="failed to get container status \"df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9\": rpc error: code = NotFound desc = could not find container \"df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9\": container with ID starting with df183aaeae2cbb49c0b61ecb72db663ab2dc4aaeb6c092df23ae594da75520a9 not found: ID does not exist" Jul 20 20:10:04.846937 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.846906 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl"] Jul 20 20:10:04.855335 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:04.855307 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-850607-openshift-default-74db579b75-47sjl"] Jul 20 20:10:05.062257 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:05.062222 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" path="/var/lib/kubelet/pods/685c9f78-02fb-4ead-b9a5-d96d2d3a65c0/volumes" Jul 20 20:10:08.245498 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.245454 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs"] Jul 20 20:10:08.246000 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.245834 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" Jul 20 20:10:08.246000 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.245845 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" Jul 20 20:10:08.246000 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.245857 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" Jul 20 20:10:08.246000 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.245863 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" Jul 20 20:10:08.246000 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.245918 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" Jul 20 20:10:08.246000 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.245925 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="685c9f78-02fb-4ead-b9a5-d96d2d3a65c0" containerName="istio-proxy" Jul 20 20:10:08.250162 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.250113 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.253260 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.253236 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-fffcf6-openshift-default-dockercfg-j6746\"" Jul 20 20:10:08.264501 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.264464 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs"] Jul 20 20:10:08.372677 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.372631 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-podinfo\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.372896 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.372745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwgbs\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-kube-api-access-lwgbs\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.372896 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.372827 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-socket\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.372896 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.372855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-token\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.372896 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.372886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-envoy\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.373050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.373015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-certs\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.373050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.373044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-data\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.373135 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.373069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-credential-socket\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.373135 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.373088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istiod-ca-cert\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.473991 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.473953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-certs\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.473991 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.473991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-data\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-credential-socket\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istiod-ca-cert\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-podinfo\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwgbs\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-kube-api-access-lwgbs\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-socket\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-token\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-envoy\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474683 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474505 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-data\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474683 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474574 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-credential-socket\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474683 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474623 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-socket\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.474937 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474841 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-certs\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.475039 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.474974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istiod-ca-cert\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.476592 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.476571 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-envoy\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.476817 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.476798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-podinfo\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.482756 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.482731 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-token\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.483602 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.483567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwgbs\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-kube-api-access-lwgbs\") pod \"e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.561581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.561540 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:08.709782 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.709749 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs"] Jul 20 20:10:08.711056 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:10:08.711028 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d84430_6ee6_43cf_8d25_c08014ce24f8.slice/crio-ff94dbb24c3bd583ef8bae6e2e77f500e7f32f124a3efe6f9cb022aee49cf829 WatchSource:0}: Error finding container ff94dbb24c3bd583ef8bae6e2e77f500e7f32f124a3efe6f9cb022aee49cf829: Status 404 returned error can't find the container with id ff94dbb24c3bd583ef8bae6e2e77f500e7f32f124a3efe6f9cb022aee49cf829 Jul 20 20:10:08.713215 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.713181 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:08.713318 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.713249 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:08.713318 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.713281 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:08.851311 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.851214 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" event={"ID":"c1d84430-6ee6-43cf-8d25-c08014ce24f8","Type":"ContainerStarted","Data":"98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48"} Jul 20 20:10:08.851311 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.851273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" event={"ID":"c1d84430-6ee6-43cf-8d25-c08014ce24f8","Type":"ContainerStarted","Data":"ff94dbb24c3bd583ef8bae6e2e77f500e7f32f124a3efe6f9cb022aee49cf829"} Jul 20 20:10:08.877558 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:08.877489 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" podStartSLOduration=0.877466885 podStartE2EDuration="877.466885ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:10:08.873117999 +0000 UTC m=+2550.498196994" watchObservedRunningTime="2026-07-20 20:10:08.877466885 +0000 UTC m=+2550.502545881" Jul 20 20:10:09.562176 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:09.562132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:09.567336 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:09.567311 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:09.856875 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:09.856786 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:09.857676 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:09.857659 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:09.949346 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:09.949308 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:10:17.845703 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:17.845654 2580 patch_prober.go:28] interesting pod/e2e-api-a-b18634-openshift-default-96f66c867-trbmw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 20:10:17.846443 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:17.846387 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 20:10:18.915420 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:18.915384 2580 generic.go:356] "Generic (PLEG): container finished" podID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerID="1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff" exitCode=0 Jul 20 20:10:18.915891 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:18.915466 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" event={"ID":"7fda4f6c-c6b2-4d22-82d6-1d8877329454","Type":"ContainerDied","Data":"1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff"} Jul 20 20:10:18.915939 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:18.915920 2580 scope.go:117] "RemoveContainer" containerID="1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff" Jul 20 20:10:18.917082 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:18.917053 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:18.917141 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:18.917133 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:18.917181 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:18.917165 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:19.922999 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:19.922950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" event={"ID":"7fda4f6c-c6b2-4d22-82d6-1d8877329454","Type":"ContainerStarted","Data":"efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983"} Jul 20 20:10:20.187941 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:20.187834 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:20.188129 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:20.188102 2580 patch_prober.go:28] interesting pod/e2e-api-a-b18634-openshift-default-96f66c867-trbmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 20:10:20.188170 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:20.188154 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 20:10:21.188132 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:21.188089 2580 patch_prober.go:28] interesting pod/e2e-api-a-b18634-openshift-default-96f66c867-trbmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 20:10:21.188646 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:21.188196 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 20:10:21.537250 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:21.537211 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:10:22.187743 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:22.187703 2580 patch_prober.go:28] interesting pod/e2e-api-a-b18634-openshift-default-96f66c867-trbmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 20:10:22.187938 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:22.187775 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 20:10:23.188012 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:23.187972 2580 patch_prober.go:28] interesting pod/e2e-api-a-b18634-openshift-default-96f66c867-trbmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 20 20:10:23.188460 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:23.188029 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 20 20:10:24.190713 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.190682 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:24.191150 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.191025 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:24.191841 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.191821 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:24.872234 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.872203 2580 patch_prober.go:28] interesting pod/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:33832->10.133.0.58:15021: read: connection reset by peer" start-of-body= Jul 20 20:10:24.872365 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.872277 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:33832->10.133.0.58:15021: read: connection reset by peer" Jul 20 20:10:24.968367 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.968332 2580 generic.go:356] "Generic (PLEG): container finished" podID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerID="98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48" exitCode=0 Jul 20 20:10:24.968542 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.968409 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" event={"ID":"c1d84430-6ee6-43cf-8d25-c08014ce24f8","Type":"ContainerDied","Data":"98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48"} Jul 20 20:10:24.968849 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.968835 2580 scope.go:117] "RemoveContainer" containerID="98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48" Jul 20 20:10:24.969812 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.969785 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:24.969873 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.969850 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:24.969923 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:24.969911 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:10:25.975559 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:25.975517 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" event={"ID":"c1d84430-6ee6-43cf-8d25-c08014ce24f8","Type":"ContainerStarted","Data":"d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22"} Jul 20 20:10:26.562760 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:26.562715 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:26.562951 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:26.562916 2580 patch_prober.go:28] interesting pod/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 20:10:26.562997 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:26.562960 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 20:10:27.562616 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:27.562572 2580 patch_prober.go:28] interesting pod/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 20:10:27.563056 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:27.562635 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 20:10:28.561990 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:28.561945 2580 patch_prober.go:28] interesting pod/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 20:10:28.562178 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:28.562011 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 20:10:29.562218 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:29.562168 2580 patch_prober.go:28] interesting pod/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 20 20:10:29.562756 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:29.562246 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 20 20:10:30.565718 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:30.565680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:30.566186 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:30.566166 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:30.566819 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:30.566798 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:10:45.630920 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:45.630880 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:10:50.288707 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:50.288670 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw"] Jul 20 20:10:50.289149 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:50.288901 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" containerID="cri-o://efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983" gracePeriod=30 Jul 20 20:10:54.194141 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:54.194101 2580 patch_prober.go:28] interesting pod/e2e-api-a-b18634-openshift-default-96f66c867-trbmw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 20:10:54.194584 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:54.194180 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 20:10:55.540058 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.540031 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:55.648571 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648468 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-data\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648571 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648511 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-socket\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648571 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648552 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-token\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648589 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-certs\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648625 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-podinfo\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648647 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istiod-ca-cert\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648685 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzslk\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-kube-api-access-wzslk\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648750 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-credential-socket\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648793 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-envoy\") pod \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\" (UID: \"7fda4f6c-c6b2-4d22-82d6-1d8877329454\") " Jul 20 20:10:55.648870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.648818 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:55.649219 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.649023 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:55.649219 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.649037 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:55.649219 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.649210 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.649366 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.649234 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.649366 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.649249 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.649538 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.649511 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:10:55.649596 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.649545 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-data" (OuterVolumeSpecName: "istio-data") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:55.651272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.651245 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-kube-api-access-wzslk" (OuterVolumeSpecName: "kube-api-access-wzslk") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "kube-api-access-wzslk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:10:55.651375 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.651253 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-token" (OuterVolumeSpecName: "istio-token") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:10:55.651375 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.651341 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:10:55.651375 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.651355 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7fda4f6c-c6b2-4d22-82d6-1d8877329454" (UID: "7fda4f6c-c6b2-4d22-82d6-1d8877329454"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:10:55.750032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.749993 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.750032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.750024 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.750032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.750034 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzslk\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-kube-api-access-wzslk\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.750032 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.750044 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.750310 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.750052 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:55.750310 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:55.750060 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fda4f6c-c6b2-4d22-82d6-1d8877329454-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:10:56.144445 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.144391 2580 generic.go:356] "Generic (PLEG): container finished" podID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerID="efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983" exitCode=0 Jul 20 20:10:56.144618 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.144453 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" Jul 20 20:10:56.144618 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.144473 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" event={"ID":"7fda4f6c-c6b2-4d22-82d6-1d8877329454","Type":"ContainerDied","Data":"efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983"} Jul 20 20:10:56.144618 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.144535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw" event={"ID":"7fda4f6c-c6b2-4d22-82d6-1d8877329454","Type":"ContainerDied","Data":"1e5f7909411f0d8f5281645443617c6245fd39d7b20a3baa430e4089563928b3"} Jul 20 20:10:56.144618 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.144559 2580 scope.go:117] "RemoveContainer" containerID="efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983" Jul 20 20:10:56.153959 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.153942 2580 scope.go:117] "RemoveContainer" containerID="1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff" Jul 20 20:10:56.162578 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.162553 2580 scope.go:117] "RemoveContainer" containerID="efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983" Jul 20 20:10:56.163039 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:10:56.163011 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983\": container with ID starting with efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983 not found: ID does not exist" containerID="efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983" Jul 20 20:10:56.163165 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.163053 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983"} err="failed to get container status \"efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983\": rpc error: code = NotFound desc = could not find container \"efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983\": container with ID starting with efad55f580d154458a881525c5fbd7e5e8bba76e30955c2bdd4505701435e983 not found: ID does not exist" Jul 20 20:10:56.163165 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.163078 2580 scope.go:117] "RemoveContainer" containerID="1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff" Jul 20 20:10:56.163467 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:10:56.163440 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff\": container with ID starting with 1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff not found: ID does not exist" containerID="1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff" Jul 20 20:10:56.163580 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.163477 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff"} err="failed to get container status \"1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff\": rpc error: code = NotFound desc = could not find container \"1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff\": container with ID starting with 1d743186c8b7c6a7e6c96e197a3d92ac0a526072301e923ebd2cd57839c5ceff not found: ID does not exist" Jul 20 20:10:56.165912 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.165890 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw"] Jul 20 20:10:56.170225 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:56.170196 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b18634-openshift-default-96f66c867-trbmw"] Jul 20 20:10:57.061911 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:10:57.061879 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" path="/var/lib/kubelet/pods/7fda4f6c-c6b2-4d22-82d6-1d8877329454/volumes" Jul 20 20:11:10.829367 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:10.829334 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:11:17.290862 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:17.290827 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs"] Jul 20 20:11:17.291355 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:17.291087 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" containerID="cri-o://d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22" gracePeriod=30 Jul 20 20:11:18.271973 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.271937 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5"] Jul 20 20:11:18.272476 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.272455 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" Jul 20 20:11:18.272584 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.272478 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" Jul 20 20:11:18.272647 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.272614 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" Jul 20 20:11:18.272647 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.272632 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" Jul 20 20:11:18.275864 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.275838 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.278594 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.278571 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-9305d0-openshift-default-dockercfg-kmncn\"" Jul 20 20:11:18.292034 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.292004 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5"] Jul 20 20:11:18.372010 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.371954 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e220a73a-e681-4b96-9125-02320cc73adc-istiod-ca-cert\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372215 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqnc7\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-kube-api-access-tqnc7\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372215 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-istio-token\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372215 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-socket\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372215 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-certs\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372215 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e220a73a-e681-4b96-9125-02320cc73adc-istio-podinfo\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372552 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-envoy\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372552 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-credential-socket\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.372552 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.372296 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-data\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.473775 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473734 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqnc7\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-kube-api-access-tqnc7\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.473775 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-istio-token\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473826 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-socket\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-certs\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e220a73a-e681-4b96-9125-02320cc73adc-istio-podinfo\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-envoy\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-credential-socket\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.473937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-data\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.474007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e220a73a-e681-4b96-9125-02320cc73adc-istiod-ca-cert\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474380 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.474336 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-socket\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474466 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.474396 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-certs\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474565 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.474539 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-data\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474744 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.474723 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-credential-socket\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.474942 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.474919 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e220a73a-e681-4b96-9125-02320cc73adc-istiod-ca-cert\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.476391 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.476372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-envoy\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.476539 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.476524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e220a73a-e681-4b96-9125-02320cc73adc-istio-podinfo\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.487948 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.487918 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-istio-token\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.488140 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.488119 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqnc7\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-kube-api-access-tqnc7\") pod \"e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.586941 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.586896 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:18.737925 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.737894 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5"] Jul 20 20:11:18.739083 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:11:18.739051 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode220a73a_e681_4b96_9125_02320cc73adc.slice/crio-6594c8a66a5c7ccc7282a1a6d52c5ac3ba2d71bf3c3952477bcbcac463859e69 WatchSource:0}: Error finding container 6594c8a66a5c7ccc7282a1a6d52c5ac3ba2d71bf3c3952477bcbcac463859e69: Status 404 returned error can't find the container with id 6594c8a66a5c7ccc7282a1a6d52c5ac3ba2d71bf3c3952477bcbcac463859e69 Jul 20 20:11:18.741289 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.741260 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:18.741368 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.741324 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:18.741368 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:18.741351 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:19.281446 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:19.281392 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" event={"ID":"e220a73a-e681-4b96-9125-02320cc73adc","Type":"ContainerStarted","Data":"b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2"} Jul 20 20:11:19.281621 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:19.281491 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" event={"ID":"e220a73a-e681-4b96-9125-02320cc73adc","Type":"ContainerStarted","Data":"6594c8a66a5c7ccc7282a1a6d52c5ac3ba2d71bf3c3952477bcbcac463859e69"} Jul 20 20:11:19.300807 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:19.300746 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podStartSLOduration=1.3007306779999999 podStartE2EDuration="1.300730678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:11:19.299134389 +0000 UTC m=+2620.924213383" watchObservedRunningTime="2026-07-20 20:11:19.300730678 +0000 UTC m=+2620.925809672" Jul 20 20:11:19.587242 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:19.587204 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:19.592352 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:19.592321 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:20.287528 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:20.287492 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:20.288501 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:20.288476 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:22.556757 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.556730 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:11:22.616532 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616437 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-certs\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.616532 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616520 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-podinfo\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.616736 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616557 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwgbs\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-kube-api-access-lwgbs\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.616736 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616587 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-token\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.616736 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616708 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istiod-ca-cert\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.616884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616762 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:11:22.616884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616770 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-data\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.616884 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616827 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-socket\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.617049 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616949 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-envoy\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.617049 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.616975 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-credential-socket\") pod \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\" (UID: \"c1d84430-6ee6-43cf-8d25-c08014ce24f8\") " Jul 20 20:11:22.617157 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.617058 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:11:22.617212 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.617150 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:11:22.617365 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.617341 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:11:22.617459 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.617351 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.617459 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.617396 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.617459 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.617412 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.617655 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.617630 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:11:22.618908 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.618883 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:11:22.618908 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.618900 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:11:22.619055 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.618995 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:11:22.619055 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.619032 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-kube-api-access-lwgbs" (OuterVolumeSpecName: "kube-api-access-lwgbs") pod "c1d84430-6ee6-43cf-8d25-c08014ce24f8" (UID: "c1d84430-6ee6-43cf-8d25-c08014ce24f8"). InnerVolumeSpecName "kube-api-access-lwgbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:11:22.717892 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.717851 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.717892 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.717881 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwgbs\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-kube-api-access-lwgbs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.717892 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.717891 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.717892 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.717901 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.718160 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.717910 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:22.718160 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:22.717917 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1d84430-6ee6-43cf-8d25-c08014ce24f8-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:11:23.305221 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.305182 2580 generic.go:356] "Generic (PLEG): container finished" podID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerID="d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22" exitCode=0 Jul 20 20:11:23.305395 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.305231 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" Jul 20 20:11:23.305395 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.305257 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" event={"ID":"c1d84430-6ee6-43cf-8d25-c08014ce24f8","Type":"ContainerDied","Data":"d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22"} Jul 20 20:11:23.305395 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.305306 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs" event={"ID":"c1d84430-6ee6-43cf-8d25-c08014ce24f8","Type":"ContainerDied","Data":"ff94dbb24c3bd583ef8bae6e2e77f500e7f32f124a3efe6f9cb022aee49cf829"} Jul 20 20:11:23.305395 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.305332 2580 scope.go:117] "RemoveContainer" containerID="d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22" Jul 20 20:11:23.315206 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.315180 2580 scope.go:117] "RemoveContainer" containerID="98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48" Jul 20 20:11:23.324071 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.324047 2580 scope.go:117] "RemoveContainer" containerID="d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22" Jul 20 20:11:23.324390 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:11:23.324367 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22\": container with ID starting with d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22 not found: ID does not exist" containerID="d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22" Jul 20 20:11:23.324520 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.324398 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22"} err="failed to get container status \"d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22\": rpc error: code = NotFound desc = could not find container \"d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22\": container with ID starting with d6fb343ebed92a234143c8fd16aea4476b22958749af4ef3b7d87955e4d57b22 not found: ID does not exist" Jul 20 20:11:23.324520 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.324418 2580 scope.go:117] "RemoveContainer" containerID="98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48" Jul 20 20:11:23.324727 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:11:23.324705 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48\": container with ID starting with 98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48 not found: ID does not exist" containerID="98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48" Jul 20 20:11:23.324777 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.324733 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48"} err="failed to get container status \"98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48\": rpc error: code = NotFound desc = could not find container \"98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48\": container with ID starting with 98d8e140b351fa19f4fed66ddeeaa6f60e69f751e971c49e68535476dadc2e48 not found: ID does not exist" Jul 20 20:11:23.329393 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.329365 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs"] Jul 20 20:11:23.335700 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:23.335669 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-fffcf6-openshift-default-6cf7d5fb8c-gj9cs"] Jul 20 20:11:25.062187 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:25.062152 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" path="/var/lib/kubelet/pods/c1d84430-6ee6-43cf-8d25-c08014ce24f8/volumes" Jul 20 20:11:26.535133 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.535092 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:11:26.652765 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.652725 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42"] Jul 20 20:11:26.653138 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.653123 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" Jul 20 20:11:26.653138 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.653138 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" Jul 20 20:11:26.653259 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.653149 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" Jul 20 20:11:26.653259 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.653154 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" Jul 20 20:11:26.653259 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.653161 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" Jul 20 20:11:26.653259 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.653169 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fda4f6c-c6b2-4d22-82d6-1d8877329454" containerName="istio-proxy" Jul 20 20:11:26.653259 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.653222 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" Jul 20 20:11:26.657849 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.657813 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.660023 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.659996 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b02932-openshift-default-dockercfg-fttmr\"" Jul 20 20:11:26.673034 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.673003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42"] Jul 20 20:11:26.756049 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756008 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khzhn\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-kube-api-access-khzhn\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.756381 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-certs\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.756586 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-envoy\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.756734 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756719 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10236152-0a2d-44ad-92ff-7c73b59db459-istio-podinfo\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.756886 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-istio-token\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.757017 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10236152-0a2d-44ad-92ff-7c73b59db459-istiod-ca-cert\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.757017 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-socket\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.757017 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.756966 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-data\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.757017 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.757011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-credential-socket\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858331 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khzhn\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-kube-api-access-khzhn\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858331 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-certs\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858331 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-envoy\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858717 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10236152-0a2d-44ad-92ff-7c73b59db459-istio-podinfo\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858717 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-istio-token\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858717 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10236152-0a2d-44ad-92ff-7c73b59db459-istiod-ca-cert\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858717 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-socket\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858717 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-data\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.858717 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-credential-socket\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.859019 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.858890 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-certs\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.859116 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.859091 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-credential-socket\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.859183 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.859154 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-data\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.859229 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.859197 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-socket\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.859357 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.859332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10236152-0a2d-44ad-92ff-7c73b59db459-istiod-ca-cert\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.860786 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.860763 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-envoy\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.861262 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.861242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10236152-0a2d-44ad-92ff-7c73b59db459-istio-podinfo\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.866281 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.866253 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-istio-token\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.866386 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.866332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khzhn\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-kube-api-access-khzhn\") pod \"e2e-shared-b-b02932-openshift-default-85b886b779-l9h42\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:26.968629 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:26.968586 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:27.123611 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.123579 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42"] Jul 20 20:11:27.125882 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:11:27.125835 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10236152_0a2d_44ad_92ff_7c73b59db459.slice/crio-8b51e3e5066894bb74f89b3d9274c30f7392a1df7d41334614b945cbfb404306 WatchSource:0}: Error finding container 8b51e3e5066894bb74f89b3d9274c30f7392a1df7d41334614b945cbfb404306: Status 404 returned error can't find the container with id 8b51e3e5066894bb74f89b3d9274c30f7392a1df7d41334614b945cbfb404306 Jul 20 20:11:27.128207 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.128164 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:27.128304 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.128236 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:27.128304 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.128265 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:27.331997 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.331950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" event={"ID":"10236152-0a2d-44ad-92ff-7c73b59db459","Type":"ContainerStarted","Data":"d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593"} Jul 20 20:11:27.332215 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.332010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" event={"ID":"10236152-0a2d-44ad-92ff-7c73b59db459","Type":"ContainerStarted","Data":"8b51e3e5066894bb74f89b3d9274c30f7392a1df7d41334614b945cbfb404306"} Jul 20 20:11:27.355976 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.355912 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" podStartSLOduration=1.355894668 podStartE2EDuration="1.355894668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:11:27.353683803 +0000 UTC m=+2628.978762806" watchObservedRunningTime="2026-07-20 20:11:27.355894668 +0000 UTC m=+2628.980973662" Jul 20 20:11:27.969622 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:27.969577 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:28.973970 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:28.973934 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:29.351052 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:29.350947 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:29.352477 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:29.352406 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:35.289556 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.289518 2580 patch_prober.go:28] interesting pod/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 20:11:35.289982 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.289592 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 20:11:35.386214 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.386178 2580 generic.go:356] "Generic (PLEG): container finished" podID="e220a73a-e681-4b96-9125-02320cc73adc" containerID="b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2" exitCode=0 Jul 20 20:11:35.386385 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.386251 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" event={"ID":"e220a73a-e681-4b96-9125-02320cc73adc","Type":"ContainerDied","Data":"b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2"} Jul 20 20:11:35.386733 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.386717 2580 scope.go:117] "RemoveContainer" containerID="b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2" Jul 20 20:11:35.387751 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.387725 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:35.387824 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.387790 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:35.387868 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.387827 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:35.526270 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:35.526232 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:11:36.394223 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:36.394184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" event={"ID":"e220a73a-e681-4b96-9125-02320cc73adc","Type":"ContainerStarted","Data":"af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69"} Jul 20 20:11:36.587679 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:36.587644 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:36.588310 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:36.587956 2580 patch_prober.go:28] interesting pod/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 20:11:36.588310 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:36.588083 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 20:11:37.588310 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:37.588269 2580 patch_prober.go:28] interesting pod/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 20:11:37.588746 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:37.588327 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 20:11:38.588246 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:38.588211 2580 patch_prober.go:28] interesting pod/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 20:11:38.588525 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:38.588501 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 20:11:39.588142 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:39.588106 2580 patch_prober.go:28] interesting pod/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 20 20:11:39.588344 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:39.588164 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 20 20:11:40.592085 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:40.592055 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:40.592533 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:40.592517 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:40.593196 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:40.593174 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:11:43.435900 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:43.435866 2580 generic.go:356] "Generic (PLEG): container finished" podID="10236152-0a2d-44ad-92ff-7c73b59db459" containerID="d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593" exitCode=0 Jul 20 20:11:43.436326 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:43.435942 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" event={"ID":"10236152-0a2d-44ad-92ff-7c73b59db459","Type":"ContainerDied","Data":"d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593"} Jul 20 20:11:43.436373 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:43.436342 2580 scope.go:117] "RemoveContainer" containerID="d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593" Jul 20 20:11:43.437464 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:43.437408 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:43.437564 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:43.437526 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:43.437564 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:43.437560 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 20 20:11:44.351271 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:44.351228 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:44.447658 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:44.447612 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" event={"ID":"10236152-0a2d-44ad-92ff-7c73b59db459","Type":"ContainerStarted","Data":"320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368"} Jul 20 20:11:44.969270 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:44.969226 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:44.970447 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:44.970397 2580 patch_prober.go:28] interesting pod/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 20 20:11:44.970586 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:44.970468 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 20 20:11:45.969872 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:45.969822 2580 patch_prober.go:28] interesting pod/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 20 20:11:45.970457 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:45.969905 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 20 20:11:46.969683 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:46.969643 2580 patch_prober.go:28] interesting pod/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 20 20:11:46.969876 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:46.969717 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 20 20:11:47.969482 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:47.969447 2580 patch_prober.go:28] interesting pod/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 20 20:11:47.969888 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:47.969503 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 20 20:11:48.973113 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:48.973080 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:48.973529 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:48.973376 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:11:48.974074 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:11:48.974055 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:12:30.025756 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:12:30.025710 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:12:58.229028 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:12:58.228989 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:13:15.024097 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:13:15.024055 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:13:33.434677 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:13:33.434636 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:13:49.726044 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:13:49.726002 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:14:07.533298 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:14:07.533211 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:14:24.733390 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:14:24.733350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:14:41.928879 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:14:41.928840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:15:00.141692 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:15:00.141654 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:15:13.633012 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:15:13.632973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:15:26.231036 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:15:26.231000 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:15:47.932322 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:15:47.932234 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:16:06.138234 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:16:06.138195 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:16:23.336740 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:16:23.336705 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:16:32.531653 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:16:32.531610 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:16:53.933510 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:16:53.933472 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:17:19.235631 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:17:19.235600 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:17:34.430843 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:17:34.430802 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:17:53.635140 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:17:53.635099 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:18:14.029774 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:14.029741 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:18:22.669338 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:22.669300 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5"] Jul 20 20:18:22.669901 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:22.669562 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" containerID="cri-o://af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69" gracePeriod=30 Jul 20 20:18:25.594841 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:25.594797 2580 patch_prober.go:28] interesting pod/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 20:18:25.595305 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:25.594860 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 20:18:27.921656 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:27.921624 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:18:28.038302 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038270 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-certs\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038547 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038325 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-data\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038547 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038346 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e220a73a-e681-4b96-9125-02320cc73adc-istio-podinfo\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038547 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038417 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e220a73a-e681-4b96-9125-02320cc73adc-istiod-ca-cert\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038547 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038507 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-istio-token\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038547 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-socket\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038823 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038568 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-envoy\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038823 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038604 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqnc7\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-kube-api-access-tqnc7\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038823 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038635 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-credential-socket\") pod \"e220a73a-e681-4b96-9125-02320cc73adc\" (UID: \"e220a73a-e681-4b96-9125-02320cc73adc\") " Jul 20 20:18:28.038823 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038668 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:28.039016 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038836 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:28.039016 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.038854 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e220a73a-e681-4b96-9125-02320cc73adc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:18:28.039122 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.039023 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e220a73a-e681-4b96-9125-02320cc73adc-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.039122 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.039041 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.039122 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.039053 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.039122 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.039063 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:28.039471 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.039423 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-data" (OuterVolumeSpecName: "istio-data") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:28.040753 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.040725 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e220a73a-e681-4b96-9125-02320cc73adc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:18:28.040862 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.040785 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-istio-token" (OuterVolumeSpecName: "istio-token") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:18:28.040862 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.040792 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-kube-api-access-tqnc7" (OuterVolumeSpecName: "kube-api-access-tqnc7") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "kube-api-access-tqnc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:18:28.040943 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.040918 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e220a73a-e681-4b96-9125-02320cc73adc" (UID: "e220a73a-e681-4b96-9125-02320cc73adc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:28.140512 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.140471 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.140512 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.140504 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e220a73a-e681-4b96-9125-02320cc73adc-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.140512 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.140513 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.140512 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.140523 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.140865 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.140531 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqnc7\" (UniqueName: \"kubernetes.io/projected/e220a73a-e681-4b96-9125-02320cc73adc-kube-api-access-tqnc7\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.140865 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.140540 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e220a73a-e681-4b96-9125-02320cc73adc-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:28.673675 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.673636 2580 generic.go:356] "Generic (PLEG): container finished" podID="e220a73a-e681-4b96-9125-02320cc73adc" containerID="af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69" exitCode=0 Jul 20 20:18:28.673867 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.673689 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" Jul 20 20:18:28.673867 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.673689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" event={"ID":"e220a73a-e681-4b96-9125-02320cc73adc","Type":"ContainerDied","Data":"af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69"} Jul 20 20:18:28.673867 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.673817 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5" event={"ID":"e220a73a-e681-4b96-9125-02320cc73adc","Type":"ContainerDied","Data":"6594c8a66a5c7ccc7282a1a6d52c5ac3ba2d71bf3c3952477bcbcac463859e69"} Jul 20 20:18:28.673867 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.673844 2580 scope.go:117] "RemoveContainer" containerID="af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69" Jul 20 20:18:28.684328 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.684304 2580 scope.go:117] "RemoveContainer" containerID="b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2" Jul 20 20:18:28.693555 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.693531 2580 scope.go:117] "RemoveContainer" containerID="af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69" Jul 20 20:18:28.693908 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:18:28.693883 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69\": container with ID starting with af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69 not found: ID does not exist" containerID="af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69" Jul 20 20:18:28.693968 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.693916 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69"} err="failed to get container status \"af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69\": rpc error: code = NotFound desc = could not find container \"af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69\": container with ID starting with af99c4a8afa4f432ad3f810addc627f0cb75e37b6420f2fc33da0eac396a1d69 not found: ID does not exist" Jul 20 20:18:28.693968 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.693936 2580 scope.go:117] "RemoveContainer" containerID="b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2" Jul 20 20:18:28.694209 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:18:28.694189 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2\": container with ID starting with b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2 not found: ID does not exist" containerID="b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2" Jul 20 20:18:28.694263 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.694217 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2"} err="failed to get container status \"b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2\": rpc error: code = NotFound desc = could not find container \"b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2\": container with ID starting with b0f1f2a403e5bf9cc0669926a4fb7b792d2dcf71dfbd1e90207676f9b72eabd2 not found: ID does not exist" Jul 20 20:18:28.699243 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.699215 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5"] Jul 20 20:18:28.702418 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:28.702392 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-9305d0-openshift-default-59dc976dbc-f5dc5"] Jul 20 20:18:29.061619 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:29.061583 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e220a73a-e681-4b96-9125-02320cc73adc" path="/var/lib/kubelet/pods/e220a73a-e681-4b96-9125-02320cc73adc/volumes" Jul 20 20:18:36.631493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:36.631387 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lwnr4"] Jul 20 20:18:44.961792 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:44.961753 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42"] Jul 20 20:18:44.962276 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:44.961970 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" containerID="cri-o://320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368" gracePeriod=30 Jul 20 20:18:48.976198 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:48.976154 2580 patch_prober.go:28] interesting pod/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 20:18:48.976615 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:48.976238 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 20:18:50.208249 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.208214 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:18:50.363493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363458 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-certs\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.363699 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363529 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-envoy\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.363699 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363558 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-credential-socket\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.363699 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363583 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khzhn\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-kube-api-access-khzhn\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.363699 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363606 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-data\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.363699 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363639 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-istio-token\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.363699 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363697 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10236152-0a2d-44ad-92ff-7c73b59db459-istio-podinfo\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.364050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363715 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10236152-0a2d-44ad-92ff-7c73b59db459-istiod-ca-cert\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.364050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363765 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-socket\") pod \"10236152-0a2d-44ad-92ff-7c73b59db459\" (UID: \"10236152-0a2d-44ad-92ff-7c73b59db459\") " Jul 20 20:18:50.364050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363822 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:50.364050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.363925 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:50.364050 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.364042 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:50.364314 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.364134 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-certs\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.364314 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.364151 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-credential-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.364314 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.364196 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-workload-socket\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.364497 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.364315 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10236152-0a2d-44ad-92ff-7c73b59db459-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 20:18:50.364706 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.364679 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-data" (OuterVolumeSpecName: "istio-data") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:50.365913 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.365888 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-kube-api-access-khzhn" (OuterVolumeSpecName: "kube-api-access-khzhn") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "kube-api-access-khzhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:18:50.366070 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.366046 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-istio-token" (OuterVolumeSpecName: "istio-token") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:18:50.366213 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.366183 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10236152-0a2d-44ad-92ff-7c73b59db459-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 20:18:50.366213 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.366183 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10236152-0a2d-44ad-92ff-7c73b59db459" (UID: "10236152-0a2d-44ad-92ff-7c73b59db459"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:18:50.465663 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.465634 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-envoy\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.465663 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.465662 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khzhn\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-kube-api-access-khzhn\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.465663 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.465672 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10236152-0a2d-44ad-92ff-7c73b59db459-istio-data\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.465959 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.465681 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10236152-0a2d-44ad-92ff-7c73b59db459-istio-token\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.465959 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.465689 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10236152-0a2d-44ad-92ff-7c73b59db459-istio-podinfo\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.465959 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.465696 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10236152-0a2d-44ad-92ff-7c73b59db459-istiod-ca-cert\") on node \"ip-10-0-132-219.ec2.internal\" DevicePath \"\"" Jul 20 20:18:50.795269 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.795233 2580 generic.go:356] "Generic (PLEG): container finished" podID="10236152-0a2d-44ad-92ff-7c73b59db459" containerID="320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368" exitCode=0 Jul 20 20:18:50.795464 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.795283 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" Jul 20 20:18:50.795464 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.795300 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" event={"ID":"10236152-0a2d-44ad-92ff-7c73b59db459","Type":"ContainerDied","Data":"320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368"} Jul 20 20:18:50.795464 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.795348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42" event={"ID":"10236152-0a2d-44ad-92ff-7c73b59db459","Type":"ContainerDied","Data":"8b51e3e5066894bb74f89b3d9274c30f7392a1df7d41334614b945cbfb404306"} Jul 20 20:18:50.795464 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.795370 2580 scope.go:117] "RemoveContainer" containerID="320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368" Jul 20 20:18:50.805717 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.805697 2580 scope.go:117] "RemoveContainer" containerID="d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593" Jul 20 20:18:50.814017 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.813995 2580 scope.go:117] "RemoveContainer" containerID="320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368" Jul 20 20:18:50.814284 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:18:50.814265 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368\": container with ID starting with 320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368 not found: ID does not exist" containerID="320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368" Jul 20 20:18:50.814353 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.814291 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368"} err="failed to get container status \"320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368\": rpc error: code = NotFound desc = could not find container \"320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368\": container with ID starting with 320340fa5f047c6afd8aac407d6d35119390d8f8ab387b8cc45dd251d1d97368 not found: ID does not exist" Jul 20 20:18:50.814353 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.814310 2580 scope.go:117] "RemoveContainer" containerID="d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593" Jul 20 20:18:50.814595 ip-10-0-132-219 kubenswrapper[2580]: E0720 20:18:50.814573 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593\": container with ID starting with d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593 not found: ID does not exist" containerID="d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593" Jul 20 20:18:50.814657 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.814600 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593"} err="failed to get container status \"d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593\": rpc error: code = NotFound desc = could not find container \"d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593\": container with ID starting with d8c674dc020c323e77fab843059313b52287cb062ca0aa939da4faa4f6048593 not found: ID does not exist" Jul 20 20:18:50.817751 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.817728 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42"] Jul 20 20:18:50.822971 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:50.822949 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b02932-openshift-default-85b886b779-l9h42"] Jul 20 20:18:51.062066 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:51.061978 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" path="/var/lib/kubelet/pods/10236152-0a2d-44ad-92ff-7c73b59db459/volumes" Jul 20 20:18:51.568103 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:51.568073 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xjpq8_375cbf6c-bbc2-46f6-8711-ea94b37ec6ff/manager/0.log" Jul 20 20:18:51.703930 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:51.703903 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gbdwj_c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7/manager/0.log" Jul 20 20:18:51.962980 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:51.962898 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hz8g8_b46b79d3-e102-409f-ad51-3e3376954fc6/server/0.log" Jul 20 20:18:52.094834 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:52.094794 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gxzbw_871baf92-5f83-4e69-a5af-6aba4bb234d3/manager/0.log" Jul 20 20:18:52.355119 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:52.355090 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-v4vzj_7ac466b5-52bf-4284-808d-7281f3e59775/copy-manifests/0.log" Jul 20 20:18:52.363827 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:52.363792 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-v4vzj_7ac466b5-52bf-4284-808d-7281f3e59775/manager/0.log" Jul 20 20:18:54.230265 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.230231 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v_0627a651-c27a-4990-bc5f-1b82ad647d95/util/0.log" Jul 20 20:18:54.237849 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.237825 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v_0627a651-c27a-4990-bc5f-1b82ad647d95/pull/0.log" Jul 20 20:18:54.252735 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.252709 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v_0627a651-c27a-4990-bc5f-1b82ad647d95/extract/0.log" Jul 20 20:18:54.373057 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.373028 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6_7143dbdb-4aaf-447d-8980-7554bb1249cc/util/0.log" Jul 20 20:18:54.379701 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.379664 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6_7143dbdb-4aaf-447d-8980-7554bb1249cc/pull/0.log" Jul 20 20:18:54.386311 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.386279 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6_7143dbdb-4aaf-447d-8980-7554bb1249cc/extract/0.log" Jul 20 20:18:54.510299 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.510218 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6_d18c2e80-fd73-4678-8374-3ad5cd6538fd/util/0.log" Jul 20 20:18:54.517581 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.517540 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6_d18c2e80-fd73-4678-8374-3ad5cd6538fd/pull/0.log" Jul 20 20:18:54.525071 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.525053 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6_d18c2e80-fd73-4678-8374-3ad5cd6538fd/extract/0.log" Jul 20 20:18:54.648583 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.648553 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4_28a07d8c-af38-46d3-b1b3-1f778349da03/util/0.log" Jul 20 20:18:54.656443 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.656403 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4_28a07d8c-af38-46d3-b1b3-1f778349da03/pull/0.log" Jul 20 20:18:54.663692 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.663670 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4_28a07d8c-af38-46d3-b1b3-1f778349da03/extract/0.log" Jul 20 20:18:54.931499 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:54.931465 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-t8k6g_9d71df69-24d2-49b0-bba9-e5ff4238e93b/manager/0.log" Jul 20 20:18:55.297469 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:55.297415 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2mzkh_722323c6-cb0a-4baa-920e-927667b7bc89/registry-server/0.log" Jul 20 20:18:55.557919 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:55.557838 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-lwnr4_a8a809ab-20b8-4554-a765-960d6d810403/limitador/0.log" Jul 20 20:18:55.692460 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:55.691694 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7gkqr_6954ccee-eaea-402a-8894-49e29808da89/manager/0.log" Jul 20 20:18:56.066085 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:56.066041 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnks9_36556998-8345-4788-80b0-21dd646c950f/istio-proxy/0.log" Jul 20 20:18:56.190618 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:56.190570 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ppscm_40dcbb61-bb8e-4084-b820-4719b106c922/discovery/0.log" Jul 20 20:18:56.560769 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:56.560734 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gqgd2_091738fc-368d-424f-9fb1-f5d36400e97c/istio-proxy/1.log" Jul 20 20:18:56.942412 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:56.942318 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59844479fb-ld75n_100f1256-c5f4-4a3b-bc90-daba6644692c/router/0.log" Jul 20 20:18:57.459084 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:57.459055 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vkgfw_c91bbf27-f981-4a4b-98c3-f334fe936d20/main/0.log" Jul 20 20:18:57.585266 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:18:57.585239 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qsfr4_dd74c50e-247c-46bc-b2eb-b17cb56b3d5a/main/0.log" Jul 20 20:19:06.418118 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:06.418048 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dgq65_c1e1ab83-94d8-4a81-9d3f-c14b59e8e542/global-pull-secret-syncer/0.log" Jul 20 20:19:06.600409 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:06.600358 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fx9q6_a960bab6-ebec-418e-b780-1294ccf3b592/konnectivity-agent/0.log" Jul 20 20:19:06.685888 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:06.685783 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-219.ec2.internal_c22b57f42ff2be1a90bb5b7bd856167f/haproxy/0.log" Jul 20 20:19:11.399896 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.399863 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v_0627a651-c27a-4990-bc5f-1b82ad647d95/extract/0.log" Jul 20 20:19:11.426614 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.426584 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v_0627a651-c27a-4990-bc5f-1b82ad647d95/util/0.log" Jul 20 20:19:11.453320 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.453285 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fgw8v_0627a651-c27a-4990-bc5f-1b82ad647d95/pull/0.log" Jul 20 20:19:11.486234 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.486207 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6_7143dbdb-4aaf-447d-8980-7554bb1249cc/extract/0.log" Jul 20 20:19:11.525421 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.525391 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6_7143dbdb-4aaf-447d-8980-7554bb1249cc/util/0.log" Jul 20 20:19:11.551058 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.551034 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqvn6_7143dbdb-4aaf-447d-8980-7554bb1249cc/pull/0.log" Jul 20 20:19:11.594557 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.594528 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6_d18c2e80-fd73-4678-8374-3ad5cd6538fd/extract/0.log" Jul 20 20:19:11.621990 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.621964 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6_d18c2e80-fd73-4678-8374-3ad5cd6538fd/util/0.log" Jul 20 20:19:11.647233 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.647210 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cbdn6_d18c2e80-fd73-4678-8374-3ad5cd6538fd/pull/0.log" Jul 20 20:19:11.681164 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.681080 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4_28a07d8c-af38-46d3-b1b3-1f778349da03/extract/0.log" Jul 20 20:19:11.705894 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.705871 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4_28a07d8c-af38-46d3-b1b3-1f778349da03/util/0.log" Jul 20 20:19:11.736360 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:11.736331 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14msn4_28a07d8c-af38-46d3-b1b3-1f778349da03/pull/0.log" Jul 20 20:19:12.149556 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:12.149527 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-t8k6g_9d71df69-24d2-49b0-bba9-e5ff4238e93b/manager/0.log" Jul 20 20:19:12.259688 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:12.259642 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2mzkh_722323c6-cb0a-4baa-920e-927667b7bc89/registry-server/0.log" Jul 20 20:19:12.407533 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:12.407423 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-lwnr4_a8a809ab-20b8-4554-a765-960d6d810403/limitador/0.log" Jul 20 20:19:12.486557 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:12.486525 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7gkqr_6954ccee-eaea-402a-8894-49e29808da89/manager/0.log" Jul 20 20:19:14.515071 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:14.515040 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-n72w6_c7e7a011-b8d6-463d-9da0-f4353591ce25/monitoring-plugin/0.log" Jul 20 20:19:14.732493 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:14.732463 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q4fz9_d9b8dcfc-d329-49e4-8c22-554e1f8a6acd/node-exporter/0.log" Jul 20 20:19:14.763797 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:14.763765 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q4fz9_d9b8dcfc-d329-49e4-8c22-554e1f8a6acd/kube-rbac-proxy/0.log" Jul 20 20:19:14.787798 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:14.787766 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q4fz9_d9b8dcfc-d329-49e4-8c22-554e1f8a6acd/init-textfile/0.log" Jul 20 20:19:16.463481 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463447 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p"] Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463817 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463830 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463840 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463846 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463860 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463866 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463880 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463885 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463955 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1d84430-6ee6-43cf-8d25-c08014ce24f8" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463964 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10236152-0a2d-44ad-92ff-7c73b59db459" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463969 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" Jul 20 20:19:16.464006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.463976 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e220a73a-e681-4b96-9125-02320cc73adc" containerName="istio-proxy" Jul 20 20:19:16.467577 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.467559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.469552 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.469526 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mvcn6\"/\"default-dockercfg-66lh9\"" Jul 20 20:19:16.469688 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.469526 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mvcn6\"/\"openshift-service-ca.crt\"" Jul 20 20:19:16.469688 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.469575 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mvcn6\"/\"kube-root-ca.crt\"" Jul 20 20:19:16.476543 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.476521 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p"] Jul 20 20:19:16.595547 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.595503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-sys\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.595728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.595574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-podres\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.595728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.595647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-lib-modules\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.595728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.595689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-proc\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.595851 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.595754 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr9qc\" (UniqueName: \"kubernetes.io/projected/f31e4d72-fdf1-441c-804a-7b99fb25a536-kube-api-access-tr9qc\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696666 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-sys\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-podres\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-lib-modules\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-proc\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696779 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-sys\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr9qc\" (UniqueName: \"kubernetes.io/projected/f31e4d72-fdf1-441c-804a-7b99fb25a536-kube-api-access-tr9qc\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696844 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-proc\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.696870 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696863 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-podres\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.697219 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.696911 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f31e4d72-fdf1-441c-804a-7b99fb25a536-lib-modules\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.704358 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.704334 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr9qc\" (UniqueName: \"kubernetes.io/projected/f31e4d72-fdf1-441c-804a-7b99fb25a536-kube-api-access-tr9qc\") pod \"perf-node-gather-daemonset-fdr5p\" (UID: \"f31e4d72-fdf1-441c-804a-7b99fb25a536\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.778670 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.778576 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:16.911520 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.911491 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p"] Jul 20 20:19:16.913507 ip-10-0-132-219 kubenswrapper[2580]: W0720 20:19:16.913468 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf31e4d72_fdf1_441c_804a_7b99fb25a536.slice/crio-0ca3ceca2fa2205115940fab3e7af3e6bbab7c5754f791ff40aeab32353a695a WatchSource:0}: Error finding container 0ca3ceca2fa2205115940fab3e7af3e6bbab7c5754f791ff40aeab32353a695a: Status 404 returned error can't find the container with id 0ca3ceca2fa2205115940fab3e7af3e6bbab7c5754f791ff40aeab32353a695a Jul 20 20:19:16.939728 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:16.939692 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" event={"ID":"f31e4d72-fdf1-441c-804a-7b99fb25a536","Type":"ContainerStarted","Data":"0ca3ceca2fa2205115940fab3e7af3e6bbab7c5754f791ff40aeab32353a695a"} Jul 20 20:19:17.886540 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:17.886500 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76b9f8855f-xn2zq_da21864d-d850-448c-b254-56ecac205fe7/console/0.log" Jul 20 20:19:17.946421 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:17.946389 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" event={"ID":"f31e4d72-fdf1-441c-804a-7b99fb25a536","Type":"ContainerStarted","Data":"d5c995077841b6c5eaaa5e750d408afcac44f36c770430a07cfe59026ae254c3"} Jul 20 20:19:17.946666 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:17.946539 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:17.961994 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:17.961938 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" podStartSLOduration=1.961920986 podStartE2EDuration="1.961920986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 20:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:19:17.961750157 +0000 UTC m=+3099.586829151" watchObservedRunningTime="2026-07-20 20:19:17.961920986 +0000 UTC m=+3099.586999981" Jul 20 20:19:19.613051 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:19.613016 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tvwsq_c967fb7b-6a3f-428a-8600-a7c39c1973af/dns/0.log" Jul 20 20:19:19.635899 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:19.635872 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tvwsq_c967fb7b-6a3f-428a-8600-a7c39c1973af/kube-rbac-proxy/0.log" Jul 20 20:19:19.663145 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:19.663115 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-48ksk_40ce65db-b05a-4930-927c-b81a075191df/dns-node-resolver/0.log" Jul 20 20:19:20.288566 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:20.288519 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-dd6ffbd9-rd87d_2791edcd-bf0d-4010-8bb6-ee25f69aab36/registry/0.log" Jul 20 20:19:20.365895 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:20.365868 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmf8z_5fa9cad5-2584-4972-9ab0-47dd94309bdc/node-ca/0.log" Jul 20 20:19:21.308147 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:21.308098 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnks9_36556998-8345-4788-80b0-21dd646c950f/istio-proxy/0.log" Jul 20 20:19:21.436849 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:21.436817 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ppscm_40dcbb61-bb8e-4084-b820-4719b106c922/discovery/0.log" Jul 20 20:19:21.529864 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:21.529838 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gqgd2_091738fc-368d-424f-9fb1-f5d36400e97c/istio-proxy/0.log" Jul 20 20:19:21.556437 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:21.556408 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gqgd2_091738fc-368d-424f-9fb1-f5d36400e97c/istio-proxy/1.log" Jul 20 20:19:21.649853 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:21.649767 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59844479fb-ld75n_100f1256-c5f4-4a3b-bc90-daba6644692c/router/0.log" Jul 20 20:19:22.336621 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:22.336588 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qmn6p_ce9b176e-f8c6-45de-bd20-fc9f8b9cf013/serve-healthcheck-canary/0.log" Jul 20 20:19:22.825224 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:22.825193 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-qqmfb_ec1ccec2-98b0-435d-9e8e-c2de5706aadd/insights-operator/0.log" Jul 20 20:19:22.825584 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:22.825564 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-qqmfb_ec1ccec2-98b0-435d-9e8e-c2de5706aadd/insights-operator/1.log" Jul 20 20:19:22.849821 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:22.849796 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rdtp_bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd/kube-rbac-proxy/0.log" Jul 20 20:19:22.872002 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:22.871971 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rdtp_bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd/exporter/0.log" Jul 20 20:19:22.894418 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:22.894387 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rdtp_bbe760d1-64f1-4424-9a7d-cf3c6aa5a8dd/extractor/0.log" Jul 20 20:19:23.964909 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:23.964880 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-fdr5p" Jul 20 20:19:25.444110 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:25.444070 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xjpq8_375cbf6c-bbc2-46f6-8711-ea94b37ec6ff/manager/0.log" Jul 20 20:19:25.639005 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:25.638972 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gbdwj_c02f3ea0-0f4e-42dd-9fe0-4ca4f3059aa7/manager/0.log" Jul 20 20:19:25.831562 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:25.831534 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hz8g8_b46b79d3-e102-409f-ad51-3e3376954fc6/server/0.log" Jul 20 20:19:25.903572 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:25.903502 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gxzbw_871baf92-5f83-4e69-a5af-6aba4bb234d3/manager/0.log" Jul 20 20:19:25.991039 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:25.991002 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-v4vzj_7ac466b5-52bf-4284-808d-7281f3e59775/manager/0.log" Jul 20 20:19:26.018826 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:26.018798 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-v4vzj_7ac466b5-52bf-4284-808d-7281f3e59775/copy-manifests/0.log" Jul 20 20:19:27.226890 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:27.226863 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnks9_36556998-8345-4788-80b0-21dd646c950f/istio-proxy/0.log" Jul 20 20:19:27.342162 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:27.342127 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ppscm_40dcbb61-bb8e-4084-b820-4719b106c922/discovery/0.log" Jul 20 20:19:27.397806 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:27.397773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gqgd2_091738fc-368d-424f-9fb1-f5d36400e97c/istio-proxy/0.log" Jul 20 20:19:27.425002 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:27.424976 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gqgd2_091738fc-368d-424f-9fb1-f5d36400e97c/istio-proxy/1.log" Jul 20 20:19:27.492779 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:27.492696 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59844479fb-ld75n_100f1256-c5f4-4a3b-bc90-daba6644692c/router/0.log" Jul 20 20:19:27.528006 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:27.527983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wsq74_86eb4aaf-d9de-4f34-91e1-0404b3d4cc1e/openshift-lws-operator/0.log" Jul 20 20:19:28.767893 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:28.767847 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vnks9_36556998-8345-4788-80b0-21dd646c950f/istio-proxy/0.log" Jul 20 20:19:28.890212 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:28.890177 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ppscm_40dcbb61-bb8e-4084-b820-4719b106c922/discovery/0.log" Jul 20 20:19:28.942320 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:28.942280 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gqgd2_091738fc-368d-424f-9fb1-f5d36400e97c/istio-proxy/0.log" Jul 20 20:19:28.968963 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:28.968933 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gqgd2_091738fc-368d-424f-9fb1-f5d36400e97c/istio-proxy/1.log" Jul 20 20:19:29.039419 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:29.039394 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59844479fb-ld75n_100f1256-c5f4-4a3b-bc90-daba6644692c/router/0.log" Jul 20 20:19:33.009481 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:33.009447 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-75wvc_ef772c49-d191-4c78-b4fe-13512691a076/kube-storage-version-migrator-operator/1.log" Jul 20 20:19:33.010272 ip-10-0-132-219 kubenswrapper[2580]: I0720 20:19:33.010254 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-75wvc_ef772c49-d191-4c78-b4fe-13512691a076/kube-storage-version-migrator-operator/0.log"