Aug 04 00:00:24.875289 ip-10-0-138-239 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Aug 04 00:00:24.875300 ip-10-0-138-239 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Aug 04 00:00:24.875306 ip-10-0-138-239 systemd[1]: kubelet.service: Failed with result 'resources'. Aug 04 00:00:24.875512 ip-10-0-138-239 systemd[1]: Failed to start Kubernetes Kubelet. Aug 04 00:00:34.941283 ip-10-0-138-239 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Aug 04 00:00:34.941303 ip-10-0-138-239 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3273083064b14d3f91fb5e4cf26c637f -- Aug 04 00:02:12.307252 ip-10-0-138-239 systemd[1]: Starting Kubernetes Kubelet... Aug 04 00:02:12.751111 ip-10-0-138-239 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 00:02:12.751111 ip-10-0-138-239 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 00:02:12.751111 ip-10-0-138-239 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 00:02:12.751111 ip-10-0-138-239 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 00:02:12.751111 ip-10-0-138-239 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 00:02:12.753656 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.753571 2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 00:02:12.758789 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758769 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 00:02:12.758789 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758785 2551 feature_gate.go:328] unrecognized feature gate: Example Aug 04 00:02:12.758789 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758790 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 00:02:12.758789 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758794 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758800 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758804 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758808 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758813 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758817 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758821 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758825 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758831 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758837 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758841 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758845 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758848 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758852 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758856 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758870 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758874 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758879 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758883 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 00:02:12.759063 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758887 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758891 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758895 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758899 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758909 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758913 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758917 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758921 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758926 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758930 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758934 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758938 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758943 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758947 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758951 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758957 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758962 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758966 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758984 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 00:02:12.759667 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758988 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758992 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.758997 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759001 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759006 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759010 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759014 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759019 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759023 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759028 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759043 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759049 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759054 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759058 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759063 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759067 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759072 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759077 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759081 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 00:02:12.760142 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759086 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759090 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759094 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759098 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759103 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759107 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759111 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759115 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759120 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759125 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759129 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759134 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759138 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759143 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759147 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759151 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759156 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759160 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759164 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759168 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 00:02:12.760630 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759173 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759177 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759181 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759186 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759190 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759195 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759199 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759203 2551 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759209 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759213 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759218 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759222 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759226 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759231 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759235 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759242 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759248 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759253 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 00:02:12.761134 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.759258 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 00:02:12.762139 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762124 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 00:02:12.762139 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762137 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 00:02:12.762139 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762143 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762147 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762152 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762156 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762161 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762165 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762169 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762174 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762178 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762181 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762185 2551 feature_gate.go:328] unrecognized feature gate: Example Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762189 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762193 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762197 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762201 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762207 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762213 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762216 2551 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762220 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 00:02:12.762312 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762225 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762229 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762233 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762236 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762241 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762245 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762249 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762253 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762258 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762262 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762266 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762270 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762274 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762281 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762287 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762294 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762298 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762303 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762307 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 00:02:12.763157 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762311 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762315 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762320 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762324 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762328 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762333 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762337 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762341 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762345 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762351 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762355 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762359 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762363 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762367 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762372 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762376 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762380 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762384 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762388 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762392 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762397 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 00:02:12.763997 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762401 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762405 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762409 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762413 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762417 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762421 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762425 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762431 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762435 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762440 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762444 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762448 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762452 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762456 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762461 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762465 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762469 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762474 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762478 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 00:02:12.764907 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762483 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762487 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762491 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762495 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762501 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762505 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762509 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762514 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762518 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762522 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762529 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762534 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762538 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762543 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762547 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762552 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762556 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762560 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.762564 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 00:02:12.765533 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763164 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763173 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763178 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763183 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763187 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763192 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763196 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763200 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763206 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763210 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763214 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763242 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763249 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763254 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763259 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763264 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763268 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763273 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763277 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 00:02:12.766023 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763281 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763286 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763290 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763295 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763299 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763303 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763308 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763311 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763316 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763320 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763324 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763329 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763333 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763338 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763343 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763347 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763352 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763356 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763361 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763365 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 00:02:12.766547 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763369 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763373 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763378 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763383 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763387 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763391 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763397 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763401 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763406 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763411 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763415 2551 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763419 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763424 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763428 2551 feature_gate.go:328] unrecognized feature gate: Example Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763432 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763436 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763440 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763444 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763448 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763452 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 00:02:12.767155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763457 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763465 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763470 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763475 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763479 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763483 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763488 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763493 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763497 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763501 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763508 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763514 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763519 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763524 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763528 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763533 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763537 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763542 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763546 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 00:02:12.767829 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763551 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763558 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763563 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763568 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763572 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763577 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763582 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763586 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763591 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763596 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763600 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763605 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763609 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763614 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763618 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763622 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763626 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763630 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763635 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 00:02:12.768310 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763640 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.763644 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763735 2551 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763745 2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763753 2551 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763760 2551 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763766 2551 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763771 2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763777 2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763784 2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763790 2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763797 2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763805 2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763810 2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763815 2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763820 2551 flags.go:64] FLAG: --cgroup-root="" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763825 2551 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763830 2551 flags.go:64] FLAG: --client-ca-file="" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763834 2551 flags.go:64] FLAG: --cloud-provider="external" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763839 2551 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763846 2551 flags.go:64] FLAG: --cluster-domain="" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763850 2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763855 2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763860 2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 00:02:12.768807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763866 2551 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763872 2551 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763876 2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763881 2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763886 2551 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763891 2551 flags.go:64] FLAG: --contention-profiling="false" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763895 2551 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763900 2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763906 2551 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763911 2551 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763918 2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763922 2551 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763927 2551 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763932 2551 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763936 2551 flags.go:64] FLAG: --enable-server="true" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763940 2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763947 2551 flags.go:64] FLAG: --event-burst="100" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763952 2551 flags.go:64] FLAG: --event-qps="50" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763956 2551 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763962 2551 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763967 2551 flags.go:64] FLAG: --eviction-hard="" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763989 2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763993 2551 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.763998 2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764002 2551 flags.go:64] FLAG: --eviction-soft="" Aug 04 00:02:12.769488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764007 2551 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764011 2551 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764016 2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764021 2551 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764025 2551 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764030 2551 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764034 2551 flags.go:64] FLAG: --feature-gates="" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764040 2551 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764045 2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764050 2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764055 2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764059 2551 flags.go:64] FLAG: --healthz-port="10248" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764064 2551 flags.go:64] FLAG: --help="false" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764070 2551 flags.go:64] FLAG: --hostname-override="ip-10-0-138-239.ec2.internal" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764074 2551 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764082 2551 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764087 2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764093 2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764098 2551 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764103 2551 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764107 2551 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764112 2551 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764116 2551 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 00:02:12.770179 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764120 2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764126 2551 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764131 2551 flags.go:64] FLAG: --kube-reserved="" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764135 2551 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764152 2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764157 2551 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764161 2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764166 2551 flags.go:64] FLAG: --lock-file="" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764170 2551 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764175 2551 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764180 2551 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764194 2551 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764199 2551 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764203 2551 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764208 2551 flags.go:64] FLAG: --logging-format="text" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764214 2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764219 2551 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764224 2551 flags.go:64] FLAG: --manifest-url="" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764228 2551 flags.go:64] FLAG: --manifest-url-header="" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764235 2551 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764239 2551 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764245 2551 flags.go:64] FLAG: --max-pods="110" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764250 2551 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764255 2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764264 2551 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 00:02:12.770827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764268 2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764273 2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764278 2551 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764283 2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764294 2551 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764300 2551 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764305 2551 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764310 2551 flags.go:64] FLAG: --pod-cidr="" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764314 2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764323 2551 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764327 2551 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764332 2551 flags.go:64] FLAG: --pods-per-core="0" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764344 2551 flags.go:64] FLAG: --port="10250" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764350 2551 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764354 2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-004bfce94c92d8b20" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764359 2551 flags.go:64] FLAG: --qos-reserved="" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764364 2551 flags.go:64] FLAG: --read-only-port="10255" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764369 2551 flags.go:64] FLAG: --register-node="true" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764374 2551 flags.go:64] FLAG: --register-with-taints="" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764380 2551 flags.go:64] FLAG: --registry-burst="10" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764385 2551 flags.go:64] FLAG: --registry-qps="5" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764389 2551 flags.go:64] FLAG: --reserved-cpus="" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764394 2551 flags.go:64] FLAG: --reserved-memory="" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764399 2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764404 2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 00:02:12.771468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764409 2551 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764413 2551 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764418 2551 flags.go:64] FLAG: --runonce="false" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764423 2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764428 2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764433 2551 flags.go:64] FLAG: --seccomp-default="false" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764440 2551 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764445 2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764450 2551 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764455 2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764460 2551 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764465 2551 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764469 2551 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764474 2551 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764478 2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764483 2551 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764488 2551 flags.go:64] FLAG: --system-cgroups="" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764493 2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764501 2551 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764506 2551 flags.go:64] FLAG: --tls-cert-file="" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764518 2551 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764529 2551 flags.go:64] FLAG: --tls-min-version="" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764533 2551 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764537 2551 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764542 2551 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 00:02:12.772171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764547 2551 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764552 2551 flags.go:64] FLAG: --v="2" Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764558 2551 flags.go:64] FLAG: --version="false" Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764564 2551 flags.go:64] FLAG: --vmodule="" Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764569 2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.764574 2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764750 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764757 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764761 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764766 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764770 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764774 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764778 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764785 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764793 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764797 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764801 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764806 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764810 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764815 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764819 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 00:02:12.772778 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764823 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764827 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764832 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764836 2551 feature_gate.go:328] unrecognized feature gate: Example Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764841 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764845 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764849 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764854 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764858 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764863 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764867 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764871 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764876 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764880 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764884 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764888 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764892 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764896 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764901 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764905 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 00:02:12.773324 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764909 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764913 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764917 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764921 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764927 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764934 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764938 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764942 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764947 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764951 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764955 2551 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764960 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764964 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764983 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764988 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764992 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.764999 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765004 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765008 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 00:02:12.773836 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765013 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765018 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765022 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765027 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765033 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765039 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765043 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765048 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765052 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765056 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765060 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765065 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765069 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765074 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765078 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765082 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765086 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765093 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765098 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 00:02:12.774316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765103 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765107 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765111 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765116 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765120 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765124 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765128 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765132 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765136 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765141 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765145 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765150 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765154 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765158 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765165 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765171 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765176 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765180 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765184 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765189 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 00:02:12.774772 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765193 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765197 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765201 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765205 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765209 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.765214 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.766179 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.773532 2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.773547 2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.773562 2551 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.773571 2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773625 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773630 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773633 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773636 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 00:02:12.775279 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773639 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773642 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773645 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773648 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773651 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773653 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773656 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773659 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773662 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773664 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773667 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773670 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773673 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773676 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773678 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773681 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773684 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773687 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773689 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773692 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 00:02:12.775657 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773694 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773697 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773700 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773703 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773706 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773709 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773712 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773714 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773717 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773719 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773722 2551 feature_gate.go:328] unrecognized feature gate: Example Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773725 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773727 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773731 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773733 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773736 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773740 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773743 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773745 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773748 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 00:02:12.776155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773751 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773753 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773756 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773759 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773761 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773766 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773770 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773773 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773776 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773778 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773781 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773784 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773786 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773789 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773792 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773796 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773800 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773803 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773806 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 00:02:12.776637 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773809 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773812 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773815 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773818 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773822 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773825 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773828 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773831 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773834 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773837 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773840 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773843 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773846 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773849 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773852 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773855 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773858 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773860 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773863 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 00:02:12.777162 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773865 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773868 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773871 2551 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773873 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773876 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773878 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773881 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773884 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773886 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773889 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773892 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773895 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773897 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773900 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773903 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773905 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 00:02:12.777618 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.773908 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.773913 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774048 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774054 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774058 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774061 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774064 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774067 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774069 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774072 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774075 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774077 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774080 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774082 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774085 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 00:02:12.778022 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774088 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774090 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774093 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774096 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774098 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774101 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774103 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774105 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774108 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774111 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774114 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774117 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774120 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774122 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774125 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774128 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774130 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774133 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774135 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774138 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 00:02:12.778385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774140 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774143 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774146 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774149 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774151 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774154 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774156 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774159 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774161 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774164 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774167 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774169 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774172 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774174 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774177 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774180 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774183 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774186 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774190 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 00:02:12.778848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774193 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774196 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774200 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774203 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774206 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774208 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774211 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774213 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774217 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774220 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774223 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774226 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774230 2551 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774233 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774236 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774239 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774241 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774244 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774247 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 00:02:12.779389 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774249 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774252 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774256 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774259 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774262 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774264 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774267 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774270 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774273 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774275 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774278 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774281 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774284 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774286 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774289 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774291 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774294 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774296 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774300 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 00:02:12.779848 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774302 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774305 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774307 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774309 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774312 2551 feature_gate.go:328] unrecognized feature gate: Example Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774314 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774317 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774320 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:12.774323 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.774327 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.774418 2551 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.776814 2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.778655 2551 server.go:1019] "Starting client certificate rotation" Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.778748 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 00:02:12.780316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.778784 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 00:02:12.782268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.782248 2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 00:02:12.783673 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.783660 2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 00:02:12.792962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.792944 2551 log.go:25] "Validated CRI v1 runtime API" Aug 04 00:02:12.796746 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.796729 2551 log.go:25] "Validated CRI v1 image API" Aug 04 00:02:12.797561 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.797541 2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 00:02:12.802375 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.802349 2551 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 84733b81-8595-4030-af4c-6f1c04043907:/dev/nvme0n1p4 97565326-b8fb-4845-a115-f21d40de68b6:/dev/nvme0n1p3] Aug 04 00:02:12.802446 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.802375 2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 00:02:12.806836 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.806820 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 00:02:12.808360 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.808261 2551 manager.go:216] Machine: {Timestamp:2026-08-04 00:02:12.805821671 +0000 UTC m=+0.374093024 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097759 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec278741441ba70f9fe5be74529abae4 SystemUUID:ec278741-441b-a70f-9fe5-be74529abae4 BootID:32730830-64b1-4d3f-91fb-5e4cf26c637f Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:82:04:26:b5:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:82:04:26:b5:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:8b:48:6a:ff:f1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 00:02:12.808360 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.808359 2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 00:02:12.808469 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.808431 2551 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 00:02:12.811832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.811805 2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 00:02:12.811982 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.811834 2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-239.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 00:02:12.813402 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.813392 2551 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 00:02:12.813439 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.813405 2551 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 00:02:12.813439 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.813415 2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 00:02:12.814274 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.814264 2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 00:02:12.815528 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.815518 2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 00:02:12.820826 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.820815 2551 state_mem.go:36] "Initialized new in-memory state store" Aug 04 00:02:12.820933 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.820924 2551 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 00:02:12.820963 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.820953 2551 kubelet.go:486] "Attempting to sync node with API server" Aug 04 00:02:12.820963 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.820962 2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 00:02:12.821032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.820989 2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 00:02:12.821032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.820999 2551 kubelet.go:398] "Adding apiserver pod source" Aug 04 00:02:12.821032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.821008 2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 00:02:12.821294 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.821284 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 00:02:12.821328 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.821301 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 00:02:12.824163 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.824145 2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 00:02:12.825777 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.825762 2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 00:02:12.825829 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.825792 2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 00:02:12.827153 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827143 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 00:02:12.827203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827157 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 00:02:12.827203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827171 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 00:02:12.827203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827176 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 00:02:12.827203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827185 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 00:02:12.827203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827190 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 00:02:12.827203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827198 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 00:02:12.827203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827203 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 00:02:12.827379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827209 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 00:02:12.827379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827214 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 00:02:12.827379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827225 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 00:02:12.827379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.827231 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 00:02:12.828078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.828066 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 00:02:12.828114 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.828080 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 00:02:12.829338 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.829322 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nkqq8" Aug 04 00:02:12.830404 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.830383 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 00:02:12.830505 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.830488 2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-239.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 00:02:12.830593 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.830576 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-239.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 00:02:12.831277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.831263 2551 server.go:1268] "Started kubelet" Aug 04 00:02:12.831436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.831346 2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 00:02:12.831530 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.831456 2551 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 00:02:12.831586 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.831402 2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 00:02:12.832016 ip-10-0-138-239 systemd[1]: Started Kubernetes Kubelet. Aug 04 00:02:12.833009 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.832992 2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 00:02:12.836694 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.836671 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nkqq8" Aug 04 00:02:12.837082 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.837062 2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 00:02:12.837364 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.837350 2551 server.go:310] "Adding debug handlers to kubelet server" Aug 04 00:02:12.840428 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.839328 2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-239.ec2.internal.18c87136e4aeb6c7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-239.ec2.internal,UID:ip-10-0-138-239.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-239.ec2.internal,},FirstTimestamp:2026-08-04 00:02:12.831229639 +0000 UTC m=+0.399500995,LastTimestamp:2026-08-04 00:02:12.831229639 +0000 UTC m=+0.399500995,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-239.ec2.internal,}" Aug 04 00:02:12.843654 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.843636 2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 00:02:12.843740 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.843662 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 00:02:12.843740 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.843718 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:12.845436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.844413 2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 00:02:12.845436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.844412 2551 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 00:02:12.845436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.844439 2551 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 00:02:12.845699 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.845666 2551 reconstruct.go:97] "Volume reconstruction finished" Aug 04 00:02:12.845699 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.845677 2551 reconciler.go:29] "Reconciler: start to sync state" Aug 04 00:02:12.845823 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.845712 2551 factory.go:55] Registering systemd factory Aug 04 00:02:12.845823 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.845727 2551 factory.go:223] Registration of the systemd container factory successfully Aug 04 00:02:12.845934 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.845919 2551 factory.go:153] Registering CRI-O factory Aug 04 00:02:12.845934 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.845933 2551 factory.go:223] Registration of the crio container factory successfully Aug 04 00:02:12.846091 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.845994 2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 00:02:12.846091 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.846017 2551 factory.go:103] Registering Raw factory Aug 04 00:02:12.846091 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.846031 2551 manager.go:1196] Started watching for new ooms in manager Aug 04 00:02:12.846508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.846496 2551 manager.go:349] Starting recovery of all containers Aug 04 00:02:12.853067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.853048 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 00:02:12.856637 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.856135 2551 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-239.ec2.internal\" not found" node="ip-10-0-138-239.ec2.internal" Aug 04 00:02:12.856637 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.856655 2551 manager.go:354] Recovery completed Aug 04 00:02:12.860947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.860934 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 00:02:12.862735 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.862721 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientMemory" Aug 04 00:02:12.862821 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.862754 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 00:02:12.862821 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.862769 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientPID" Aug 04 00:02:12.863410 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.863393 2551 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 00:02:12.863410 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.863409 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 00:02:12.863512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.863409 2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 00:02:12.863512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.863502 2551 state_mem.go:36] "Initialized new in-memory state store" Aug 04 00:02:12.864545 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.864521 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientMemory" Aug 04 00:02:12.864545 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.864548 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 00:02:12.864660 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.864562 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientPID" Aug 04 00:02:12.865687 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.865670 2551 policy_none.go:49] "None policy: Start" Aug 04 00:02:12.865687 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.865686 2551 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 00:02:12.865800 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.865696 2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 00:02:12.867659 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.867647 2551 policy_none.go:47] "Start" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.909473 2551 manager.go:337] "Starting Device Plugin manager" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.909617 2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.909627 2551 server.go:87] "Starting device plugin registration server" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.909850 2551 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.909860 2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.909956 2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.910049 2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.910059 2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.910488 2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 00:02:12.914832 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.910532 2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:12.963228 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.963198 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 00:02:12.964332 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.964316 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 00:02:12.964414 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.964350 2551 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 00:02:12.964414 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.964369 2551 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 00:02:12.964414 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:12.964399 2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 00:02:12.966503 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:12.966482 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 00:02:13.010136 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.010088 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 00:02:13.010926 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.010912 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientMemory" Aug 04 00:02:13.011005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.010942 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 00:02:13.011005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.010957 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientPID" Aug 04 00:02:13.011005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.010990 2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.020593 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.020578 2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.020669 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.020597 2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-239.ec2.internal\": node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:13.034583 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.034562 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:13.065082 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.065063 2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal"] Aug 04 00:02:13.065139 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.065133 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 00:02:13.066005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.065992 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientMemory" Aug 04 00:02:13.066078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.066016 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 00:02:13.066078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.066026 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientPID" Aug 04 00:02:13.068335 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068323 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 00:02:13.068465 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068450 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.068508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068481 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 00:02:13.068946 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068930 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientMemory" Aug 04 00:02:13.068946 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068945 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientMemory" Aug 04 00:02:13.069083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068954 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 00:02:13.069083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068964 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 00:02:13.069083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068983 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientPID" Aug 04 00:02:13.069083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.068990 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientPID" Aug 04 00:02:13.071316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.071300 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.071377 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.071326 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 00:02:13.071962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.071947 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientMemory" Aug 04 00:02:13.072044 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.071983 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 00:02:13.072044 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.071995 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeHasSufficientPID" Aug 04 00:02:13.085580 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.085560 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-239.ec2.internal\" not found" node="ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.089285 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.089270 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-239.ec2.internal\" not found" node="ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.134858 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.134839 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:13.146308 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.146286 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed703b30cab3073cc2ebb81efc3691e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal\" (UID: \"ed703b30cab3073cc2ebb81efc3691e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.146380 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.146314 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed703b30cab3073cc2ebb81efc3691e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal\" (UID: \"ed703b30cab3073cc2ebb81efc3691e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.146380 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.146331 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3831fba3d380769b026bc2429ce14da9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-239.ec2.internal\" (UID: \"3831fba3d380769b026bc2429ce14da9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.235323 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.235285 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:13.246626 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.246603 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed703b30cab3073cc2ebb81efc3691e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal\" (UID: \"ed703b30cab3073cc2ebb81efc3691e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.246729 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.246683 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed703b30cab3073cc2ebb81efc3691e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal\" (UID: \"ed703b30cab3073cc2ebb81efc3691e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.246729 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.246690 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed703b30cab3073cc2ebb81efc3691e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal\" (UID: \"ed703b30cab3073cc2ebb81efc3691e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.246729 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.246704 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed703b30cab3073cc2ebb81efc3691e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal\" (UID: \"ed703b30cab3073cc2ebb81efc3691e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.246841 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.246729 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3831fba3d380769b026bc2429ce14da9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-239.ec2.internal\" (UID: \"3831fba3d380769b026bc2429ce14da9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.246841 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.246763 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3831fba3d380769b026bc2429ce14da9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-239.ec2.internal\" (UID: \"3831fba3d380769b026bc2429ce14da9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.336071 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.336004 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:13.388529 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.388506 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.392395 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:13.392379 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" Aug 04 00:02:13.436340 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:13.436315 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.224631 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.224593 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.325199 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.325134 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.425715 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.425685 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.465224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.465202 2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 00:02:15.465613 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.465333 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 00:02:15.465613 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.465346 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 00:02:15.526592 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.526568 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.532026 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.531984 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-02 23:57:14 +0000 UTC" deadline="2028-04-18 04:24:09.108087456 +0000 UTC" Aug 04 00:02:15.532026 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.532025 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14956h21m53.576066535s" Aug 04 00:02:15.626935 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.626914 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.697382 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.697358 2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 00:02:15.727563 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.727527 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.745427 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:15.745396 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3831fba3d380769b026bc2429ce14da9.slice/crio-35d96a2f41133eeb8a28718236c8526f4fee0d465a2a02a3691c265b3c31ed19 WatchSource:0}: Error finding container 35d96a2f41133eeb8a28718236c8526f4fee0d465a2a02a3691c265b3c31ed19: Status 404 returned error can't find the container with id 35d96a2f41133eeb8a28718236c8526f4fee0d465a2a02a3691c265b3c31ed19 Aug 04 00:02:15.745887 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:15.745861 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded703b30cab3073cc2ebb81efc3691e7.slice/crio-82a6f44ab394deab19aad3a385abbf5155a9c545726e38f87fad31a875a6e9b9 WatchSource:0}: Error finding container 82a6f44ab394deab19aad3a385abbf5155a9c545726e38f87fad31a875a6e9b9: Status 404 returned error can't find the container with id 82a6f44ab394deab19aad3a385abbf5155a9c545726e38f87fad31a875a6e9b9 Aug 04 00:02:15.750481 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.750466 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 00:02:15.827984 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.827957 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.928533 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:15.928465 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-239.ec2.internal\" not found" Aug 04 00:02:15.993344 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:15.993324 2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 00:02:16.032205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.032182 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" Aug 04 00:02:16.042122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.042099 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 00:02:16.042931 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.042917 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" Aug 04 00:02:16.050922 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.050902 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 00:02:16.509697 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.509665 2551 apiserver.go:52] "Watching apiserver" Aug 04 00:02:16.516565 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.516541 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 00:02:16.518140 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.518123 2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 00:02:16.518899 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.518869 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-kkl4r","openshift-network-diagnostics/network-check-target-dnb4v","kube-system/global-pull-secret-syncer-6k6zs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk","openshift-cluster-node-tuning-operator/tuned-gtbh2","openshift-dns/node-resolver-b5tk9","openshift-image-registry/node-ca-pgjn4","openshift-multus/multus-additional-cni-plugins-k5rzl","openshift-network-operator/iptables-alerter-ggkf6","openshift-ovn-kubernetes/ovnkube-node-ql2gr","kube-system/konnectivity-agent-4fr52","kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal","openshift-multus/multus-z8s77"] Aug 04 00:02:16.521551 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.521529 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.523683 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.523659 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-srwlb\"" Aug 04 00:02:16.523776 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.523697 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.524041 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.524024 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 00:02:16.524135 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.524030 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 00:02:16.524300 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.524273 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 00:02:16.526040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.525949 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 00:02:16.526040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.526000 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.526040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.526027 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fhcpj\"" Aug 04 00:02:16.526291 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.526058 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:16.526291 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.526163 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 00:02:16.528141 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.528121 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.530482 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.530459 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.531007 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.530990 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sg6zn\"" Aug 04 00:02:16.531110 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.531012 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 00:02:16.531110 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.531068 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 00:02:16.531282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.531266 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 00:02:16.532101 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.532083 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 00:02:16.532424 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.532405 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 00:02:16.532811 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.532781 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.532903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.532845 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 00:02:16.532994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.532976 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cp85r\"" Aug 04 00:02:16.534976 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.534959 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 00:02:16.535209 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.535196 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 00:02:16.535322 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.535306 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.535385 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.535366 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jvbds\"" Aug 04 00:02:16.537525 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.537506 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.537637 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.537620 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 00:02:16.537783 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.537765 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6jjc9\"" Aug 04 00:02:16.537874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.537828 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 00:02:16.537874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.537863 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 00:02:16.539772 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.539753 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 00:02:16.539876 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.539836 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 00:02:16.539937 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.539904 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 00:02:16.539993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.539969 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jgv2c\"" Aug 04 00:02:16.540042 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.540009 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 00:02:16.540269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.540257 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:16.540327 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.540306 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:16.542466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542448 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-registration-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.542574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542470 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-var-lib-kubelet\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.542574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542487 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w44w\" (UniqueName: \"kubernetes.io/projected/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-kube-api-access-6w44w\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.542574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542502 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3d39dda-92c3-44bf-b400-ab782afb57c3-serviceca\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.542574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542527 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.542574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542548 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysconfig\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.542574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542573 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542596 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-modprobe-d\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542619 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-tuned\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542650 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba-agent-certs\") pod \"konnectivity-agent-4fr52\" (UID: \"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba\") " pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542676 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-kubelet-config\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542700 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysctl-conf\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542732 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-run\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542758 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-dbus\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542773 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-socket-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542807 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-sys-fs\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542830 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-hosts-file\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542850 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-tmp-dir\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.542914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542876 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542876 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3d39dda-92c3-44bf-b400-ab782afb57c3-host\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542953 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c52mj\" (UniqueName: \"kubernetes.io/projected/f3d39dda-92c3-44bf-b400-ab782afb57c3-kube-api-access-c52mj\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542977 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-device-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.542999 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543020 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-kubernetes\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543043 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpfwr\" (UniqueName: \"kubernetes.io/projected/11d2be0a-00eb-4565-95de-ce5b2c459afa-kube-api-access-bpfwr\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543085 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba-konnectivity-ca\") pod \"konnectivity-agent-4fr52\" (UID: \"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba\") " pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543106 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysctl-d\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543128 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-systemd\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543150 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-sys\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543171 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-lib-modules\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543191 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11d2be0a-00eb-4565-95de-ce5b2c459afa-tmp\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543213 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg9cx\" (UniqueName: \"kubernetes.io/projected/f38f4b93-38b8-4bc1-baac-ddc661756ae6-kube-api-access-hg9cx\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.543590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543236 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-host\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.544334 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.543634 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 00:02:16.544966 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.544948 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 00:02:16.545503 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.545221 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 00:02:16.545503 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.545293 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:16.545503 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.545369 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:16.547169 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.546819 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hftt2\"" Aug 04 00:02:16.547169 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.546879 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 00:02:16.547169 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.546884 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 00:02:16.547169 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.547122 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 00:02:16.549150 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.547468 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 00:02:16.549616 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.549596 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.553037 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.553017 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b47kz\"" Aug 04 00:02:16.553115 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.553019 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 00:02:16.553115 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.553059 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 00:02:16.566994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.566967 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kws6c" Aug 04 00:02:16.577234 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.577211 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kws6c" Aug 04 00:02:16.634255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.634230 2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 00:02:16.643422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643397 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-etc-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.643549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643435 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.643549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643468 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg9cx\" (UniqueName: \"kubernetes.io/projected/f38f4b93-38b8-4bc1-baac-ddc661756ae6-kube-api-access-hg9cx\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.643549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643492 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-host\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.643549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643510 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-hostroot\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.643549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643524 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-systemd-units\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.643549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643539 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovnkube-config\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643567 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-registration-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643573 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-host\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643587 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w44w\" (UniqueName: \"kubernetes.io/projected/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-kube-api-access-6w44w\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643632 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3d39dda-92c3-44bf-b400-ab782afb57c3-serviceca\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643658 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovnkube-script-lib\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643685 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysconfig\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643710 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643734 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643756 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643778 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-tuned\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643810 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba-agent-certs\") pod \"konnectivity-agent-4fr52\" (UID: \"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba\") " pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643836 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-kubelet-config\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.643862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643860 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysctl-conf\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643908 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-registration-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643933 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-os-release\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643948 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-run-netns\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.643951 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysctl-conf\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644005 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-sys-fs\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644173 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-sys-fs\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644177 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysconfig\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644177 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-kubelet-config\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644202 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-tmp-dir\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644240 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-socket-dir-parent\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644276 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644315 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovn-node-metrics-cert\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644369 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-kubernetes\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644411 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-system-cni-dir\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644418 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-kubernetes\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.644457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644454 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7412f5be-0769-4953-b9a0-1ca6a286c1a2-host-slash\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644478 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-tmp-dir\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644455 2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644519 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-cni-binary-copy\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644573 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3d39dda-92c3-44bf-b400-ab782afb57c3-serviceca\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644581 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-conf-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644616 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysctl-d\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644658 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-sys\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644685 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-lib-modules\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644737 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-sys\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644769 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjznz\" (UniqueName: \"kubernetes.io/projected/03fb7e55-a065-427b-9404-45db03b4293e-kube-api-access-xjznz\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644810 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-sysctl-d\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644813 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-k8s-cni-cncf-io\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644853 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-netns\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644876 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc6wb\" (UniqueName: \"kubernetes.io/projected/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-kube-api-access-qc6wb\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644828 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-lib-modules\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644913 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-systemd\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644949 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-cni-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644970 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-node-log\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.644989 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-log-socket\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645008 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwmfw\" (UniqueName: \"kubernetes.io/projected/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-kube-api-access-jwmfw\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645044 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-var-lib-kubelet\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645065 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645084 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-system-cni-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645101 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-var-lib-kubelet\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645105 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-cnibin\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645150 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-cni-multus\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645173 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-multus-certs\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645193 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-env-overrides\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645218 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645260 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-os-release\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.645297 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645308 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-etc-kubernetes\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645333 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-kubelet\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.645979 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.645367 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret podName:6bc834e8-f024-4f19-8b8f-b84b2f9d1f39 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:17.145337163 +0000 UTC m=+3.025784702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret") pod "global-pull-secret-syncer-6k6zs" (UID: "6bc834e8-f024-4f19-8b8f-b84b2f9d1f39") : object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645449 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645470 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-modprobe-d\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645486 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-run\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645511 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645535 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-cni-bin\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645541 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645558 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-kubelet\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645562 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-run\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645574 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-modprobe-d\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645580 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-ovn\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645616 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-cni-netd\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645666 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-dbus\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645693 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-socket-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645716 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-hosts-file\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645754 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-dbus\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645783 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3d39dda-92c3-44bf-b400-ab782afb57c3-host\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.646553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645827 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c52mj\" (UniqueName: \"kubernetes.io/projected/f3d39dda-92c3-44bf-b400-ab782afb57c3-kube-api-access-c52mj\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645833 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-hosts-file\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645857 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj455\" (UniqueName: \"kubernetes.io/projected/365c333a-8898-472b-8c70-21345d778a50-kube-api-access-jj455\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645865 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-socket-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645859 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3d39dda-92c3-44bf-b400-ab782afb57c3-host\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645894 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645921 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-slash\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.645944 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-device-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646051 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-device-dir\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646098 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646130 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpfwr\" (UniqueName: \"kubernetes.io/projected/11d2be0a-00eb-4565-95de-ce5b2c459afa-kube-api-access-bpfwr\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646154 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba-konnectivity-ca\") pod \"konnectivity-agent-4fr52\" (UID: \"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba\") " pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646150 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f38f4b93-38b8-4bc1-baac-ddc661756ae6-etc-selinux\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646356 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-daemon-config\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646393 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-cni-bin\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646416 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646443 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-systemd\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.647205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646466 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11d2be0a-00eb-4565-95de-ce5b2c459afa-tmp\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.648004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646490 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-cnibin\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.648004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646514 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7412f5be-0769-4953-b9a0-1ca6a286c1a2-iptables-alerter-script\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.648004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646538 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzt5s\" (UniqueName: \"kubernetes.io/projected/7412f5be-0769-4953-b9a0-1ca6a286c1a2-kube-api-access-hzt5s\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.648004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646580 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-var-lib-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.648004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646690 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-systemd\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.648004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.646737 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba-konnectivity-ca\") pod \"konnectivity-agent-4fr52\" (UID: \"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba\") " pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.648004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.647913 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11d2be0a-00eb-4565-95de-ce5b2c459afa-etc-tuned\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.648328 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.648215 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba-agent-certs\") pod \"konnectivity-agent-4fr52\" (UID: \"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba\") " pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.648930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.648887 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11d2be0a-00eb-4565-95de-ce5b2c459afa-tmp\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.650456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.650436 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w44w\" (UniqueName: \"kubernetes.io/projected/315bfe51-8619-4ecf-9e28-a1cdb0b5eebf-kube-api-access-6w44w\") pod \"node-resolver-b5tk9\" (UID: \"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf\") " pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.651036 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.651013 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg9cx\" (UniqueName: \"kubernetes.io/projected/f38f4b93-38b8-4bc1-baac-ddc661756ae6-kube-api-access-hg9cx\") pod \"aws-ebs-csi-driver-node-qvdqk\" (UID: \"f38f4b93-38b8-4bc1-baac-ddc661756ae6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.655648 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.655626 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c52mj\" (UniqueName: \"kubernetes.io/projected/f3d39dda-92c3-44bf-b400-ab782afb57c3-kube-api-access-c52mj\") pod \"node-ca-pgjn4\" (UID: \"f3d39dda-92c3-44bf-b400-ab782afb57c3\") " pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.655823 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.655764 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpfwr\" (UniqueName: \"kubernetes.io/projected/11d2be0a-00eb-4565-95de-ce5b2c459afa-kube-api-access-bpfwr\") pod \"tuned-gtbh2\" (UID: \"11d2be0a-00eb-4565-95de-ce5b2c459afa\") " pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.657097 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.657050 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" event={"ID":"3831fba3d380769b026bc2429ce14da9","Type":"ContainerStarted","Data":"35d96a2f41133eeb8a28718236c8526f4fee0d465a2a02a3691c265b3c31ed19"} Aug 04 00:02:16.658245 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.658222 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" event={"ID":"ed703b30cab3073cc2ebb81efc3691e7","Type":"ContainerStarted","Data":"82a6f44ab394deab19aad3a385abbf5155a9c545726e38f87fad31a875a6e9b9"} Aug 04 00:02:16.729170 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.729139 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 00:02:16.747785 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747762 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-os-release\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747805 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-run-netns\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747831 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-socket-dir-parent\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747851 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747859 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-os-release\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747871 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovn-node-metrics-cert\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747912 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-system-cni-dir\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747919 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-socket-dir-parent\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.747942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747939 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7412f5be-0769-4953-b9a0-1ca6a286c1a2-host-slash\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747937 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-run-netns\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747966 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747989 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7412f5be-0769-4953-b9a0-1ca6a286c1a2-host-slash\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747994 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-cni-binary-copy\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.747974 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-system-cni-dir\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748019 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-conf-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748046 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjznz\" (UniqueName: \"kubernetes.io/projected/03fb7e55-a065-427b-9404-45db03b4293e-kube-api-access-xjznz\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748072 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-k8s-cni-cncf-io\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748091 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-conf-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748137 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-netns\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748137 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-k8s-cni-cncf-io\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748097 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-netns\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748180 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc6wb\" (UniqueName: \"kubernetes.io/projected/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-kube-api-access-qc6wb\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748196 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-systemd\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748214 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-cni-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748255 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-systemd\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.748302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748274 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-cni-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748373 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-node-log\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748409 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-log-socket\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748435 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwmfw\" (UniqueName: \"kubernetes.io/projected/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-kube-api-access-jwmfw\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748463 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748472 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-node-log\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748488 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-system-cni-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748512 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-cnibin\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748537 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-cni-multus\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748559 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-cni-binary-copy\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748561 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-multus-certs\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748583 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-system-cni-dir\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748475 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-log-socket\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748599 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-run-multus-certs\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748611 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-env-overrides\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748624 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-cnibin\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748641 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-cni-multus\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748654 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-os-release\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.749005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748677 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-etc-kubernetes\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748700 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-kubelet\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748715 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-etc-kubernetes\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748714 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-os-release\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748731 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748749 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-kubelet\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748754 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-cni-bin\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748777 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-kubelet\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748812 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-ovn\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748835 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-cni-netd\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748838 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-cni-bin\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748856 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-host-var-lib-kubelet\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748867 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj455\" (UniqueName: \"kubernetes.io/projected/365c333a-8898-472b-8c70-21345d778a50-kube-api-access-jj455\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748879 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-run-ovn\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748890 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-cni-netd\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748893 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748934 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-slash\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.748964 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:16.749834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.748969 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-daemon-config\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.749030 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs podName:03fb7e55-a065-427b-9404-45db03b4293e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:17.249013307 +0000 UTC m=+3.129460825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs") pod "network-metrics-daemon-kkl4r" (UID: "03fb7e55-a065-427b-9404-45db03b4293e") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749040 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-env-overrides\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749053 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-slash\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749059 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-cni-bin\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749087 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749116 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-cnibin\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749134 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-cni-bin\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749142 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7412f5be-0769-4953-b9a0-1ca6a286c1a2-iptables-alerter-script\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749158 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749167 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzt5s\" (UniqueName: \"kubernetes.io/projected/7412f5be-0769-4953-b9a0-1ca6a286c1a2-kube-api-access-hzt5s\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749183 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-cnibin\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749279 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749296 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-var-lib-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749330 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-var-lib-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749337 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-etc-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.750422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749366 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749391 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-hostroot\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749414 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-systemd-units\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749419 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-etc-openvswitch\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749438 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovnkube-config\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749442 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-hostroot\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749443 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-host-run-ovn-kubernetes\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749465 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovnkube-script-lib\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749477 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-multus-daemon-config\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749494 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749478 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-systemd-units\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749621 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749635 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/365c333a-8898-472b-8c70-21345d778a50-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749654 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.749701 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7412f5be-0769-4953-b9a0-1ca6a286c1a2-iptables-alerter-script\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.750011 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.750079 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovnkube-script-lib\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.750097 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/365c333a-8898-472b-8c70-21345d778a50-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.751613 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.750509 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovnkube-config\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.751613 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.750849 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-ovn-node-metrics-cert\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.755269 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.755239 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 00:02:16.755269 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.755262 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 00:02:16.755269 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.755274 2551 projected.go:196] Error preparing data for projected volume kube-api-access-928pd for pod openshift-network-diagnostics/network-check-target-dnb4v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:16.755505 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:16.755370 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd podName:5e843f12-703e-48d6-9e00-a121b25098f3 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:17.255355766 +0000 UTC m=+3.135803285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-928pd" (UniqueName: "kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd") pod "network-check-target-dnb4v" (UID: "5e843f12-703e-48d6-9e00-a121b25098f3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:16.757951 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.757929 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzt5s\" (UniqueName: \"kubernetes.io/projected/7412f5be-0769-4953-b9a0-1ca6a286c1a2-kube-api-access-hzt5s\") pod \"iptables-alerter-ggkf6\" (UID: \"7412f5be-0769-4953-b9a0-1ca6a286c1a2\") " pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.758037 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.758007 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj455\" (UniqueName: \"kubernetes.io/projected/365c333a-8898-472b-8c70-21345d778a50-kube-api-access-jj455\") pod \"multus-additional-cni-plugins-k5rzl\" (UID: \"365c333a-8898-472b-8c70-21345d778a50\") " pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:16.758121 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.758100 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjznz\" (UniqueName: \"kubernetes.io/projected/03fb7e55-a065-427b-9404-45db03b4293e-kube-api-access-xjznz\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:16.758547 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.758528 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwmfw\" (UniqueName: \"kubernetes.io/projected/f7cd3a94-8f13-4637-9de8-53f7b91bc1d3-kube-api-access-jwmfw\") pod \"ovnkube-node-ql2gr\" (UID: \"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3\") " pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.758698 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.758677 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc6wb\" (UniqueName: \"kubernetes.io/projected/ce5fdfb9-29b8-4469-9e7c-04b7c958b452-kube-api-access-qc6wb\") pod \"multus-z8s77\" (UID: \"ce5fdfb9-29b8-4469-9e7c-04b7c958b452\") " pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.833281 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.833200 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pgjn4" Aug 04 00:02:16.838838 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.838819 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:16.849929 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.849905 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" Aug 04 00:02:16.853448 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.853424 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" Aug 04 00:02:16.858996 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.858977 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b5tk9" Aug 04 00:02:16.866410 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.866393 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ggkf6" Aug 04 00:02:16.871875 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.871857 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8s77" Aug 04 00:02:16.877538 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.877522 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:16.883076 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:16.883059 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" Aug 04 00:02:17.151969 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.151894 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:17.152113 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.152033 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:17.152113 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.152104 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret podName:6bc834e8-f024-4f19-8b8f-b84b2f9d1f39 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:18.152086469 +0000 UTC m=+4.032533987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret") pod "global-pull-secret-syncer-6k6zs" (UID: "6bc834e8-f024-4f19-8b8f-b84b2f9d1f39") : object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:17.199864 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.199833 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7cd3a94_8f13_4637_9de8_53f7b91bc1d3.slice/crio-85d69d4fbf0eafd3d65319e3217ae4145a32f2d9f019c4576bc2303ba751b149 WatchSource:0}: Error finding container 85d69d4fbf0eafd3d65319e3217ae4145a32f2d9f019c4576bc2303ba751b149: Status 404 returned error can't find the container with id 85d69d4fbf0eafd3d65319e3217ae4145a32f2d9f019c4576bc2303ba751b149 Aug 04 00:02:17.201340 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.201311 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d39dda_92c3_44bf_b400_ab782afb57c3.slice/crio-308b4061bd2096589585b2fb448c95f87d8de6e1476c514ea540e4dc52ec5765 WatchSource:0}: Error finding container 308b4061bd2096589585b2fb448c95f87d8de6e1476c514ea540e4dc52ec5765: Status 404 returned error can't find the container with id 308b4061bd2096589585b2fb448c95f87d8de6e1476c514ea540e4dc52ec5765 Aug 04 00:02:17.204053 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.204019 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315bfe51_8619_4ecf_9e28_a1cdb0b5eebf.slice/crio-74680588cb95ba3b88f0e88547f2b6ca675e93baf6e9559cc17ab657cdbc78e1 WatchSource:0}: Error finding container 74680588cb95ba3b88f0e88547f2b6ca675e93baf6e9559cc17ab657cdbc78e1: Status 404 returned error can't find the container with id 74680588cb95ba3b88f0e88547f2b6ca675e93baf6e9559cc17ab657cdbc78e1 Aug 04 00:02:17.206127 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.206107 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5fdfb9_29b8_4469_9e7c_04b7c958b452.slice/crio-368017d47ff7d8ab6a61fc2df15e72ee28dcfc43638ecf2712edc5df12799c69 WatchSource:0}: Error finding container 368017d47ff7d8ab6a61fc2df15e72ee28dcfc43638ecf2712edc5df12799c69: Status 404 returned error can't find the container with id 368017d47ff7d8ab6a61fc2df15e72ee28dcfc43638ecf2712edc5df12799c69 Aug 04 00:02:17.207569 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.207466 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7ebb7f_20a8_43dd_b7f7_dd352ed466ba.slice/crio-134d202b14a195a860284a43e8cfe7e2feef332025cd772dc822656937f5498d WatchSource:0}: Error finding container 134d202b14a195a860284a43e8cfe7e2feef332025cd772dc822656937f5498d: Status 404 returned error can't find the container with id 134d202b14a195a860284a43e8cfe7e2feef332025cd772dc822656937f5498d Aug 04 00:02:17.228938 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.228920 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38f4b93_38b8_4bc1_baac_ddc661756ae6.slice/crio-d261509d850c6fd94c9e8cde55bc47aa177fde79c2985f1af7f9bd0b720d7deb WatchSource:0}: Error finding container d261509d850c6fd94c9e8cde55bc47aa177fde79c2985f1af7f9bd0b720d7deb: Status 404 returned error can't find the container with id d261509d850c6fd94c9e8cde55bc47aa177fde79c2985f1af7f9bd0b720d7deb Aug 04 00:02:17.230455 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.230354 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7412f5be_0769_4953_b9a0_1ca6a286c1a2.slice/crio-d2886c74b36b1564bc72bb8df9e0eb08a95b9e2946649cc04c042d4e45911be9 WatchSource:0}: Error finding container d2886c74b36b1564bc72bb8df9e0eb08a95b9e2946649cc04c042d4e45911be9: Status 404 returned error can't find the container with id d2886c74b36b1564bc72bb8df9e0eb08a95b9e2946649cc04c042d4e45911be9 Aug 04 00:02:17.231893 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.231873 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d2be0a_00eb_4565_95de_ce5b2c459afa.slice/crio-0d1f79ca1ef451b4867f06af5f393be459c49cd72dfc870ed5e97dd82cb4647b WatchSource:0}: Error finding container 0d1f79ca1ef451b4867f06af5f393be459c49cd72dfc870ed5e97dd82cb4647b: Status 404 returned error can't find the container with id 0d1f79ca1ef451b4867f06af5f393be459c49cd72dfc870ed5e97dd82cb4647b Aug 04 00:02:17.232071 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:17.232055 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365c333a_8898_472b_8c70_21345d778a50.slice/crio-7364669a306e41cdca26119c4a94f0b45c55ec7bf0313257ad04d6dc8ecaa595 WatchSource:0}: Error finding container 7364669a306e41cdca26119c4a94f0b45c55ec7bf0313257ad04d6dc8ecaa595: Status 404 returned error can't find the container with id 7364669a306e41cdca26119c4a94f0b45c55ec7bf0313257ad04d6dc8ecaa595 Aug 04 00:02:17.253005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.252984 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:17.253083 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.253069 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:17.253127 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.253109 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs podName:03fb7e55-a065-427b-9404-45db03b4293e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:18.253096672 +0000 UTC m=+4.133544190 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs") pod "network-metrics-daemon-kkl4r" (UID: "03fb7e55-a065-427b-9404-45db03b4293e") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:17.354070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.354039 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:17.354233 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.354213 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 00:02:17.354299 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.354241 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 00:02:17.354299 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.354254 2551 projected.go:196] Error preparing data for projected volume kube-api-access-928pd for pod openshift-network-diagnostics/network-check-target-dnb4v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:17.354390 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:17.354314 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd podName:5e843f12-703e-48d6-9e00-a121b25098f3 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:18.3542938 +0000 UTC m=+4.234741321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-928pd" (UniqueName: "kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd") pod "network-check-target-dnb4v" (UID: "5e843f12-703e-48d6-9e00-a121b25098f3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:17.580719 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.580611 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 23:57:16 +0000 UTC" deadline="2028-05-19 22:44:46.263812816 +0000 UTC" Aug 04 00:02:17.580719 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.580666 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15718h42m28.68315177s" Aug 04 00:02:17.662000 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.661960 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ggkf6" event={"ID":"7412f5be-0769-4953-b9a0-1ca6a286c1a2","Type":"ContainerStarted","Data":"d2886c74b36b1564bc72bb8df9e0eb08a95b9e2946649cc04c042d4e45911be9"} Aug 04 00:02:17.663482 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.663448 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" event={"ID":"f38f4b93-38b8-4bc1-baac-ddc661756ae6","Type":"ContainerStarted","Data":"d261509d850c6fd94c9e8cde55bc47aa177fde79c2985f1af7f9bd0b720d7deb"} Aug 04 00:02:17.667285 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.667255 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4fr52" event={"ID":"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba","Type":"ContainerStarted","Data":"134d202b14a195a860284a43e8cfe7e2feef332025cd772dc822656937f5498d"} Aug 04 00:02:17.670858 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.670787 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b5tk9" event={"ID":"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf","Type":"ContainerStarted","Data":"74680588cb95ba3b88f0e88547f2b6ca675e93baf6e9559cc17ab657cdbc78e1"} Aug 04 00:02:17.672175 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.672151 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8s77" event={"ID":"ce5fdfb9-29b8-4469-9e7c-04b7c958b452","Type":"ContainerStarted","Data":"368017d47ff7d8ab6a61fc2df15e72ee28dcfc43638ecf2712edc5df12799c69"} Aug 04 00:02:17.674052 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.673975 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pgjn4" event={"ID":"f3d39dda-92c3-44bf-b400-ab782afb57c3","Type":"ContainerStarted","Data":"308b4061bd2096589585b2fb448c95f87d8de6e1476c514ea540e4dc52ec5765"} Aug 04 00:02:17.676024 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.675947 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"85d69d4fbf0eafd3d65319e3217ae4145a32f2d9f019c4576bc2303ba751b149"} Aug 04 00:02:17.679423 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.678783 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" event={"ID":"3831fba3d380769b026bc2429ce14da9","Type":"ContainerStarted","Data":"96e7c3fa245d55a06dc400a626481947ea1706a98b1178499bab85f042f42f3e"} Aug 04 00:02:17.680956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.680896 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerStarted","Data":"7364669a306e41cdca26119c4a94f0b45c55ec7bf0313257ad04d6dc8ecaa595"} Aug 04 00:02:17.682963 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:17.682937 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" event={"ID":"11d2be0a-00eb-4565-95de-ce5b2c459afa","Type":"ContainerStarted","Data":"0d1f79ca1ef451b4867f06af5f393be459c49cd72dfc870ed5e97dd82cb4647b"} Aug 04 00:02:18.160849 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.160790 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:18.161014 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.160976 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:18.161085 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.161039 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret podName:6bc834e8-f024-4f19-8b8f-b84b2f9d1f39 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:20.161020919 +0000 UTC m=+6.041468444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret") pod "global-pull-secret-syncer-6k6zs" (UID: "6bc834e8-f024-4f19-8b8f-b84b2f9d1f39") : object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:18.262560 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.261777 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:18.262560 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.262160 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:18.262560 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.262227 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs podName:03fb7e55-a065-427b-9404-45db03b4293e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:20.262208051 +0000 UTC m=+6.142655592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs") pod "network-metrics-daemon-kkl4r" (UID: "03fb7e55-a065-427b-9404-45db03b4293e") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:18.363475 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.362758 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:18.363475 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.363045 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 00:02:18.363475 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.363065 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 00:02:18.363475 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.363077 2551 projected.go:196] Error preparing data for projected volume kube-api-access-928pd for pod openshift-network-diagnostics/network-check-target-dnb4v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:18.363475 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.363136 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd podName:5e843f12-703e-48d6-9e00-a121b25098f3 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:20.363116753 +0000 UTC m=+6.243564287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-928pd" (UniqueName: "kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd") pod "network-check-target-dnb4v" (UID: "5e843f12-703e-48d6-9e00-a121b25098f3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:18.583181 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.583044 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 23:57:16 +0000 UTC" deadline="2028-01-21 17:05:33.285019728 +0000 UTC" Aug 04 00:02:18.583181 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.583101 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12857h3m14.701922981s" Aug 04 00:02:18.654659 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.653956 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:18.654659 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.654088 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:18.654659 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.654479 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:18.654659 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.654580 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:18.655244 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:18.655079 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:18.655244 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:18.655174 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:20.180865 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:20.180826 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:20.181333 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.180994 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:20.181333 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.181055 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret podName:6bc834e8-f024-4f19-8b8f-b84b2f9d1f39 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:24.181037096 +0000 UTC m=+10.061484630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret") pod "global-pull-secret-syncer-6k6zs" (UID: "6bc834e8-f024-4f19-8b8f-b84b2f9d1f39") : object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:20.282224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:20.282148 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:20.282409 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.282307 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:20.282409 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.282382 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs podName:03fb7e55-a065-427b-9404-45db03b4293e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:24.282363155 +0000 UTC m=+10.162810694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs") pod "network-metrics-daemon-kkl4r" (UID: "03fb7e55-a065-427b-9404-45db03b4293e") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:20.383565 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:20.383525 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:20.383740 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.383720 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 00:02:20.383812 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.383749 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 00:02:20.383812 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.383764 2551 projected.go:196] Error preparing data for projected volume kube-api-access-928pd for pod openshift-network-diagnostics/network-check-target-dnb4v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:20.383923 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.383849 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd podName:5e843f12-703e-48d6-9e00-a121b25098f3 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:24.383828258 +0000 UTC m=+10.264275799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-928pd" (UniqueName: "kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd") pod "network-check-target-dnb4v" (UID: "5e843f12-703e-48d6-9e00-a121b25098f3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:20.653417 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:20.653268 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:20.653417 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:20.653304 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:20.653417 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:20.653268 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:20.653417 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.653402 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:20.653723 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.653460 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:20.653723 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:20.653536 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:22.662553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:22.662526 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:22.663663 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:22.663155 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:22.663663 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:22.662836 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:22.663663 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:22.663266 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:22.663663 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:22.662781 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:22.663663 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:22.663337 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:24.220093 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:24.220053 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:24.220706 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.220240 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:24.220706 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.220300 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret podName:6bc834e8-f024-4f19-8b8f-b84b2f9d1f39 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:32.220281844 +0000 UTC m=+18.100729366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret") pod "global-pull-secret-syncer-6k6zs" (UID: "6bc834e8-f024-4f19-8b8f-b84b2f9d1f39") : object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:24.320978 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:24.320941 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:24.321161 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.321092 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:24.321227 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.321165 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs podName:03fb7e55-a065-427b-9404-45db03b4293e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:32.321143879 +0000 UTC m=+18.201591420 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs") pod "network-metrics-daemon-kkl4r" (UID: "03fb7e55-a065-427b-9404-45db03b4293e") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:24.422296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:24.422228 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:24.422476 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.422395 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 00:02:24.422476 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.422418 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 00:02:24.422476 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.422431 2551 projected.go:196] Error preparing data for projected volume kube-api-access-928pd for pod openshift-network-diagnostics/network-check-target-dnb4v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:24.422640 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.422493 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd podName:5e843f12-703e-48d6-9e00-a121b25098f3 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:32.422474282 +0000 UTC m=+18.302921845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-928pd" (UniqueName: "kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd") pod "network-check-target-dnb4v" (UID: "5e843f12-703e-48d6-9e00-a121b25098f3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:24.653023 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:24.652980 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:24.653172 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.653110 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:24.654503 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:24.654471 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:24.654625 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.654574 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:24.654739 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:24.654652 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:24.654739 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:24.654728 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:26.652926 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:26.652894 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:26.652926 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:26.652905 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:26.653466 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:26.652992 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:26.653466 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:26.653041 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:26.653466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:26.653077 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:26.653466 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:26.653169 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:28.652614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:28.652574 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:28.653064 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:28.652701 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:28.653064 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:28.652733 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:28.653064 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:28.652733 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:28.653064 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:28.652861 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:28.653064 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:28.652926 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:30.653022 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:30.652983 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:30.653022 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:30.653020 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:30.653544 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:30.653000 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:30.653544 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:30.653124 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:30.653544 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:30.653176 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:30.653544 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:30.653227 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:32.280265 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:32.280215 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:32.280668 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.280359 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:32.280668 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.280429 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret podName:6bc834e8-f024-4f19-8b8f-b84b2f9d1f39 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:48.280408083 +0000 UTC m=+34.160855601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret") pod "global-pull-secret-syncer-6k6zs" (UID: "6bc834e8-f024-4f19-8b8f-b84b2f9d1f39") : object "kube-system"/"original-pull-secret" not registered Aug 04 00:02:32.380933 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:32.380899 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:32.381096 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.381019 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:32.381096 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.381095 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs podName:03fb7e55-a065-427b-9404-45db03b4293e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:48.381073583 +0000 UTC m=+34.261521114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs") pod "network-metrics-daemon-kkl4r" (UID: "03fb7e55-a065-427b-9404-45db03b4293e") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:32.481393 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:32.481346 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:32.481555 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.481503 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 00:02:32.481555 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.481527 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 00:02:32.481555 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.481537 2551 projected.go:196] Error preparing data for projected volume kube-api-access-928pd for pod openshift-network-diagnostics/network-check-target-dnb4v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:32.481671 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.481583 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd podName:5e843f12-703e-48d6-9e00-a121b25098f3 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:48.481571609 +0000 UTC m=+34.362019131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-928pd" (UniqueName: "kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd") pod "network-check-target-dnb4v" (UID: "5e843f12-703e-48d6-9e00-a121b25098f3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:32.652759 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:32.652681 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:32.652759 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:32.652728 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:32.652951 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.652817 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:32.652951 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:32.652678 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:32.652951 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.652919 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:32.653086 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:32.652999 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:34.653456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:34.653420 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:34.653911 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:34.653529 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:34.653911 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:34.653599 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:34.653911 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:34.653697 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:34.653911 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:34.653737 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:34.653911 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:34.653826 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:35.720299 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.720053 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" event={"ID":"11d2be0a-00eb-4565-95de-ce5b2c459afa","Type":"ContainerStarted","Data":"b12c39c2d58e82f547fc447086df2e12a3d82cb486f8c4063d77a6201290a5ed"} Aug 04 00:02:35.721510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.721489 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" event={"ID":"f38f4b93-38b8-4bc1-baac-ddc661756ae6","Type":"ContainerStarted","Data":"170f41adbf60a20b7b2761b36b619e5443232d8f7e2f4b9279c71d29991e9b16"} Aug 04 00:02:35.722872 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.722849 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4fr52" event={"ID":"4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba","Type":"ContainerStarted","Data":"47ef5050be13e210ec9e9a7d733f385ffa4e0feb54e1b8a7a3b4d9ee96c2da07"} Aug 04 00:02:35.724271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.724250 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b5tk9" event={"ID":"315bfe51-8619-4ecf-9e28-a1cdb0b5eebf","Type":"ContainerStarted","Data":"93ba3c64ab1301910abf2e6d9c4d984be5cdd932169e8336436e9d3b09671494"} Aug 04 00:02:35.725692 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.725671 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8s77" event={"ID":"ce5fdfb9-29b8-4469-9e7c-04b7c958b452","Type":"ContainerStarted","Data":"887fbb25a513c5cf71604c2e047c5172156e741233a5731b96c8d3e7741937fc"} Aug 04 00:02:35.726952 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.726929 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pgjn4" event={"ID":"f3d39dda-92c3-44bf-b400-ab782afb57c3","Type":"ContainerStarted","Data":"2b557cc01733169f3fadf8db6757db5b6617ecb32fe6d8e17d9e357e86ae1c78"} Aug 04 00:02:35.729479 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.729460 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"c6bf93d70bb5e06290ecdc53930228cffea253d5ce385cd0ece803971cb329c6"} Aug 04 00:02:35.729580 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.729516 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"b3af706e9fc6aca01bf5a21ec70dae0690fa0bcefc7585c025191205393191bd"} Aug 04 00:02:35.729580 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.729546 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"bb83cfde5ec4ce3d3fa33576decda5bfbaaa60a776f5799460e1aa364a0e8677"} Aug 04 00:02:35.729580 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.729574 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"c37e6dde045b6a61f6bd1435698ff06a4de8183f6283c06ad1b65345c74e6107"} Aug 04 00:02:35.729686 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.729601 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"652a8102172b55daa45b21aeb65d9c85ac8d29f37fdd5fe7aabd33d307d38f4a"} Aug 04 00:02:35.729686 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.729627 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"5aecc25d8f0aef6722ec4423a872a37ae255f2263bf030e51332d67f6fc634cd"} Aug 04 00:02:35.730783 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.730762 2551 generic.go:356] "Generic (PLEG): container finished" podID="ed703b30cab3073cc2ebb81efc3691e7" containerID="35dc297a3150cf7b47c383bbb62d6610e587c1ad1458bb955177bc13373f73fc" exitCode=0 Aug 04 00:02:35.730840 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.730821 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" event={"ID":"ed703b30cab3073cc2ebb81efc3691e7","Type":"ContainerDied","Data":"35dc297a3150cf7b47c383bbb62d6610e587c1ad1458bb955177bc13373f73fc"} Aug 04 00:02:35.732175 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.732155 2551 generic.go:356] "Generic (PLEG): container finished" podID="365c333a-8898-472b-8c70-21345d778a50" containerID="5c974a53c7dcc055f2c654859ce63192c5b429bf76e115cef83275fb9ad41622" exitCode=0 Aug 04 00:02:35.732252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.732184 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerDied","Data":"5c974a53c7dcc055f2c654859ce63192c5b429bf76e115cef83275fb9ad41622"} Aug 04 00:02:35.739485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.737467 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gtbh2" podStartSLOduration=4.197526687 podStartE2EDuration="21.737451636s" totalImagesPullingTime="17.539924949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:35.737136072 +0000 UTC m=+21.617583613" watchObservedRunningTime="2026-08-04 00:02:35.737451636 +0000 UTC m=+21.617899178" Aug 04 00:02:35.739485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.738308 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-239.ec2.internal" podStartSLOduration=19.738295649 podStartE2EDuration="19.738295649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:17.694872748 +0000 UTC m=+3.575320291" watchObservedRunningTime="2026-08-04 00:02:35.738295649 +0000 UTC m=+21.618743191" Aug 04 00:02:35.751422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.751388 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pgjn4" podStartSLOduration=4.199416319 podStartE2EDuration="21.751377476s" totalImagesPullingTime="17.551961157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:35.751006632 +0000 UTC m=+21.631454171" watchObservedRunningTime="2026-08-04 00:02:35.751377476 +0000 UTC m=+21.631825015" Aug 04 00:02:35.765398 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.765365 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4fr52" podStartSLOduration=4.233019573 podStartE2EDuration="21.765353379s" totalImagesPullingTime="17.532333806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:35.764932423 +0000 UTC m=+21.645379962" watchObservedRunningTime="2026-08-04 00:02:35.765353379 +0000 UTC m=+21.645800927" Aug 04 00:02:35.800074 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.799869 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z8s77" podStartSLOduration=4.209455194 podStartE2EDuration="21.799858653s" totalImagesPullingTime="17.590403459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:35.799662469 +0000 UTC m=+21.680110009" watchObservedRunningTime="2026-08-04 00:02:35.799858653 +0000 UTC m=+21.680306193" Aug 04 00:02:35.800367 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:35.800342 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b5tk9" podStartSLOduration=4.258615706 podStartE2EDuration="21.800333569s" totalImagesPullingTime="17.541717863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:35.780908748 +0000 UTC m=+21.661356289" watchObservedRunningTime="2026-08-04 00:02:35.800333569 +0000 UTC m=+21.680781159" Aug 04 00:02:36.652573 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:36.652538 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:36.652827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:36.652538 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:36.652827 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:36.652688 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:36.652827 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:36.652765 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:36.652827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:36.652538 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:36.653049 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:36.652934 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:36.736261 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:36.736224 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" event={"ID":"ed703b30cab3073cc2ebb81efc3691e7","Type":"ContainerStarted","Data":"ef5a74e831ed24de88275a4fb9b7bd786bd0ef8b7e94030daad24bd014ac06e8"} Aug 04 00:02:36.738105 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:36.738066 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ggkf6" event={"ID":"7412f5be-0769-4953-b9a0-1ca6a286c1a2","Type":"ContainerStarted","Data":"f58afcb0ea83820720d69e27dcea8d6b547b82e171755e769e0582a784037168"} Aug 04 00:02:36.750376 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:36.750330 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-239.ec2.internal" podStartSLOduration=20.7503141 podStartE2EDuration="20.7503141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:36.749724658 +0000 UTC m=+22.630172199" watchObservedRunningTime="2026-08-04 00:02:36.7503141 +0000 UTC m=+22.630761636" Aug 04 00:02:36.763430 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:36.763386 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ggkf6" podStartSLOduration=5.235678535 podStartE2EDuration="22.763370701s" totalImagesPullingTime="17.527692166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:36.762922034 +0000 UTC m=+22.643369576" watchObservedRunningTime="2026-08-04 00:02:36.763370701 +0000 UTC m=+22.643818243" Aug 04 00:02:37.744080 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:37.744044 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"3af4ba2a518cdef231da95b55b6c31d75bc2e0e2e8da2901b15dd6d86810106b"} Aug 04 00:02:38.653348 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:38.653243 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:38.653348 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:38.653249 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:38.653662 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:38.653368 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:38.653662 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:38.653376 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:38.653662 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:38.653446 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:38.653662 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:38.653533 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:39.050607 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:39.050531 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:39.135903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:39.135866 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:39.295207 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:39.295174 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:39.295787 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:39.295763 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:39.748253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:39.748195 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:39.748739 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:39.748717 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4fr52" Aug 04 00:02:39.801963 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:39.801920 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:40.482142 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.479868 2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 00:02:40.613138 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.612990 2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T00:02:40.480058391Z","UUID":"412c208b-0c5f-409d-a50d-484860b33638","Handler":null,"Name":"","Endpoint":""} Aug 04 00:02:40.614316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.614296 2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 00:02:40.614404 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.614331 2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 00:02:40.652609 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.652585 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:40.652706 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.652586 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:40.652755 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:40.652707 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:40.652824 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.652586 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:40.652824 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:40.652777 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:40.652932 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:40.652853 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:40.751747 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.751716 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" event={"ID":"f38f4b93-38b8-4bc1-baac-ddc661756ae6","Type":"ContainerStarted","Data":"c83b195bd425bca0019452d4c874f01c806b7d202dfff43e1e1c4ffa9bf39680"} Aug 04 00:02:40.754991 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.754967 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" event={"ID":"f7cd3a94-8f13-4637-9de8-53f7b91bc1d3","Type":"ContainerStarted","Data":"10aeb432d0a902872e86cfb2a125ab040d6a7ec0ca3a12aa490b2b80bf3a8989"} Aug 04 00:02:40.755303 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.755268 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:40.756791 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.756770 2551 generic.go:356] "Generic (PLEG): container finished" podID="365c333a-8898-472b-8c70-21345d778a50" containerID="2f340d48815094fd3758f485a114f9bbc4cc9b3f7d1ac7ca6a0d8f57acebd7e4" exitCode=0 Aug 04 00:02:40.756905 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.756829 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerDied","Data":"2f340d48815094fd3758f485a114f9bbc4cc9b3f7d1ac7ca6a0d8f57acebd7e4"} Aug 04 00:02:40.769989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.769968 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:40.780489 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:40.780454 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" podStartSLOduration=9.166030243 podStartE2EDuration="26.780443043s" totalImagesPullingTime="17.6144128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:40.778913826 +0000 UTC m=+26.659361366" watchObservedRunningTime="2026-08-04 00:02:40.780443043 +0000 UTC m=+26.660890614" Aug 04 00:02:41.758964 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:41.758939 2551 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 04 00:02:41.759689 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:41.759393 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:41.780918 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:41.778445 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:42.378321 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.378096 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dnb4v"] Aug 04 00:02:42.378488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.378436 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:42.378577 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:42.378553 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:42.378722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.378696 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6k6zs"] Aug 04 00:02:42.378826 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.378813 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:42.378927 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:42.378892 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:42.382683 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.382661 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kkl4r"] Aug 04 00:02:42.382836 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.382771 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:42.382880 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:42.382866 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:42.762812 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.762769 2551 generic.go:356] "Generic (PLEG): container finished" podID="365c333a-8898-472b-8c70-21345d778a50" containerID="e118ddf114873dadd2f67c28d1f22caa976ee4bcc5e5faffb85357cc08c411bd" exitCode=0 Aug 04 00:02:42.763205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.762856 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerDied","Data":"e118ddf114873dadd2f67c28d1f22caa976ee4bcc5e5faffb85357cc08c411bd"} Aug 04 00:02:42.764987 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.764968 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" event={"ID":"f38f4b93-38b8-4bc1-baac-ddc661756ae6","Type":"ContainerStarted","Data":"445f6d36f1a09350178699855742b0ed3816e71aba78806c68ad2f757061c205"} Aug 04 00:02:42.765076 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.765012 2551 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 04 00:02:42.806971 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:42.806936 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qvdqk" podStartSLOduration=4.858266368 podStartE2EDuration="28.806923035s" totalImagesPullingTime="23.948656667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:42.806697617 +0000 UTC m=+28.687145191" watchObservedRunningTime="2026-08-04 00:02:42.806923035 +0000 UTC m=+28.687370553" Aug 04 00:02:43.493368 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:43.493337 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:02:43.652953 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:43.652926 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:43.653075 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:43.653023 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:43.770692 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:43.770657 2551 generic.go:356] "Generic (PLEG): container finished" podID="365c333a-8898-472b-8c70-21345d778a50" containerID="c4237f433bacf83a8ec3fe8710816e4036ccbf54ca8367aa05e85c410a9d2f8b" exitCode=0 Aug 04 00:02:43.771212 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:43.770734 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerDied","Data":"c4237f433bacf83a8ec3fe8710816e4036ccbf54ca8367aa05e85c410a9d2f8b"} Aug 04 00:02:44.654356 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:44.654310 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:44.654569 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:44.654419 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:44.654569 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:44.654470 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:44.654684 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:44.654590 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:45.652840 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:45.652807 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:45.653291 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:45.652906 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6k6zs" podUID="6bc834e8-f024-4f19-8b8f-b84b2f9d1f39" Aug 04 00:02:46.652697 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:46.652667 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:46.652880 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:46.652786 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnb4v" podUID="5e843f12-703e-48d6-9e00-a121b25098f3" Aug 04 00:02:46.652880 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:46.652818 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:46.653269 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:46.652924 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kkl4r" podUID="03fb7e55-a065-427b-9404-45db03b4293e" Aug 04 00:02:47.437084 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.437057 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:47.460046 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.460019 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-239.ec2.internal" event="NodeReady" Aug 04 00:02:47.460195 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.460143 2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 00:02:47.500989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.500964 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8584f9c6d5-jr94n"] Aug 04 00:02:47.506983 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.506953 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:47.517866 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.517846 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5nk69"] Aug 04 00:02:47.517982 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.517969 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.520514 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.520314 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dptb9\"" Aug 04 00:02:47.520637 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.520586 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 00:02:47.521561 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.521004 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 00:02:47.521561 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.521293 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 00:02:47.526448 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.526425 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 00:02:47.532451 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.531103 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sx2pp"] Aug 04 00:02:47.532451 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.531255 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.534136 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.534114 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9x2gl\"" Aug 04 00:02:47.534224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.534195 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 00:02:47.534295 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.534119 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 00:02:47.548043 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.548020 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8584f9c6d5-jr94n"] Aug 04 00:02:47.548120 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.548053 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5nk69"] Aug 04 00:02:47.548120 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.548076 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sx2pp"] Aug 04 00:02:47.548190 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.548149 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:47.550628 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.550609 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-74zks\"" Aug 04 00:02:47.550628 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.550625 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 00:02:47.550928 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.550655 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 00:02:47.551031 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.551012 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 00:02:47.585060 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.585028 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:47.653049 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.653017 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:47.655565 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.655551 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 00:02:47.679364 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.679339 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:47.697285 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697267 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsm2c\" (UniqueName: \"kubernetes.io/projected/2d3e5909-e3db-42e4-a285-a991937e28f5-kube-api-access-lsm2c\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.697377 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697300 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg2qx\" (UniqueName: \"kubernetes.io/projected/dba2ddf7-7e20-424b-b5a3-06389a500ab7-kube-api-access-fg2qx\") pod \"ingress-canary-sx2pp\" (UID: \"dba2ddf7-7e20-424b-b5a3-06389a500ab7\") " pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:47.697377 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697320 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d3e5909-e3db-42e4-a285-a991937e28f5-tmp-dir\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.697443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697372 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3e5909-e3db-42e4-a285-a991937e28f5-config-volume\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.697443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697407 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.697443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697436 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/288611d8-927d-40ec-b1c9-5be0092a8f6e-ca-trust-extracted\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.697533 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697458 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-trusted-ca\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.697533 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697483 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-installation-pull-secrets\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.697595 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697534 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-image-registry-private-configuration\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.697595 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697566 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-bound-sa-token\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.697654 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697595 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba2ddf7-7e20-424b-b5a3-06389a500ab7-cert\") pod \"ingress-canary-sx2pp\" (UID: \"dba2ddf7-7e20-424b-b5a3-06389a500ab7\") " pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:47.697654 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697624 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.697654 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697642 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpvj4\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-kube-api-access-tpvj4\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.697742 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.697660 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-certificates\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.732734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.732702 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:47.798956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.798901 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3e5909-e3db-42e4-a285-a991937e28f5-config-volume\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.798956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.798934 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.798956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.798954 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/288611d8-927d-40ec-b1c9-5be0092a8f6e-ca-trust-extracted\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799122 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:47.799062 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 00:02:47.799122 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:47.799076 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8584f9c6d5-jr94n: secret "image-registry-tls" not found Aug 04 00:02:47.799122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799075 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-trusted-ca\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799113 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-installation-pull-secrets\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799289 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:47.799137 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls podName:288611d8-927d-40ec-b1c9-5be0092a8f6e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:48.299115006 +0000 UTC m=+34.179562531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls") pod "image-registry-8584f9c6d5-jr94n" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e") : secret "image-registry-tls" not found Aug 04 00:02:47.799289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799165 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-image-registry-private-configuration\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799192 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-bound-sa-token\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799217 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba2ddf7-7e20-424b-b5a3-06389a500ab7-cert\") pod \"ingress-canary-sx2pp\" (UID: \"dba2ddf7-7e20-424b-b5a3-06389a500ab7\") " pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:47.799289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799245 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.799289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799273 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpvj4\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-kube-api-access-tpvj4\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799596 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799294 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/288611d8-927d-40ec-b1c9-5be0092a8f6e-ca-trust-extracted\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799596 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799305 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-certificates\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.799596 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799372 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsm2c\" (UniqueName: \"kubernetes.io/projected/2d3e5909-e3db-42e4-a285-a991937e28f5-kube-api-access-lsm2c\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.799596 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799428 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg2qx\" (UniqueName: \"kubernetes.io/projected/dba2ddf7-7e20-424b-b5a3-06389a500ab7-kube-api-access-fg2qx\") pod \"ingress-canary-sx2pp\" (UID: \"dba2ddf7-7e20-424b-b5a3-06389a500ab7\") " pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:47.799596 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799457 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d3e5909-e3db-42e4-a285-a991937e28f5-tmp-dir\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.799596 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799498 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d3e5909-e3db-42e4-a285-a991937e28f5-config-volume\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.799912 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:47.799709 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 00:02:47.799912 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799746 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d3e5909-e3db-42e4-a285-a991937e28f5-tmp-dir\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.799912 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:47.799783 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls podName:2d3e5909-e3db-42e4-a285-a991937e28f5 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:48.299763468 +0000 UTC m=+34.180210988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls") pod "dns-default-5nk69" (UID: "2d3e5909-e3db-42e4-a285-a991937e28f5") : secret "dns-default-metrics-tls" not found Aug 04 00:02:47.800061 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.799979 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-certificates\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.800196 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.800174 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-trusted-ca\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.803177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.803152 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-image-registry-private-configuration\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.803177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.803154 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-installation-pull-secrets\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.803309 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.803223 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba2ddf7-7e20-424b-b5a3-06389a500ab7-cert\") pod \"ingress-canary-sx2pp\" (UID: \"dba2ddf7-7e20-424b-b5a3-06389a500ab7\") " pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:47.808474 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.808451 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsm2c\" (UniqueName: \"kubernetes.io/projected/2d3e5909-e3db-42e4-a285-a991937e28f5-kube-api-access-lsm2c\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:47.808601 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.808521 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg2qx\" (UniqueName: \"kubernetes.io/projected/dba2ddf7-7e20-424b-b5a3-06389a500ab7-kube-api-access-fg2qx\") pod \"ingress-canary-sx2pp\" (UID: \"dba2ddf7-7e20-424b-b5a3-06389a500ab7\") " pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:47.809089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.809070 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpvj4\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-kube-api-access-tpvj4\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.809187 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.809117 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-bound-sa-token\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:47.857425 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:47.857406 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sx2pp" Aug 04 00:02:48.061147 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.061096 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sx2pp"] Aug 04 00:02:48.063603 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:48.063578 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba2ddf7_7e20_424b_b5a3_06389a500ab7.slice/crio-e9080c955027e49a1394560e7fc9a2a525d503b069a92a9fe74cb8ea6f5e01b4 WatchSource:0}: Error finding container e9080c955027e49a1394560e7fc9a2a525d503b069a92a9fe74cb8ea6f5e01b4: Status 404 returned error can't find the container with id e9080c955027e49a1394560e7fc9a2a525d503b069a92a9fe74cb8ea6f5e01b4 Aug 04 00:02:48.303933 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.303900 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:48.304100 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.303943 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:48.304100 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.304050 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 00:02:48.304100 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.304062 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 00:02:48.304100 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.304082 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8584f9c6d5-jr94n: secret "image-registry-tls" not found Aug 04 00:02:48.304271 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.304114 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls podName:2d3e5909-e3db-42e4-a285-a991937e28f5 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:49.304096098 +0000 UTC m=+35.184543617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls") pod "dns-default-5nk69" (UID: "2d3e5909-e3db-42e4-a285-a991937e28f5") : secret "dns-default-metrics-tls" not found Aug 04 00:02:48.304271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.304133 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:48.304271 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.304182 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls podName:288611d8-927d-40ec-b1c9-5be0092a8f6e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:49.30417188 +0000 UTC m=+35.184619402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls") pod "image-registry-8584f9c6d5-jr94n" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e") : secret "image-registry-tls" not found Aug 04 00:02:48.306459 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.306443 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6bc834e8-f024-4f19-8b8f-b84b2f9d1f39-original-pull-secret\") pod \"global-pull-secret-syncer-6k6zs\" (UID: \"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39\") " pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:48.405510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.405482 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:48.405671 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.405604 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:48.405671 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.405659 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs podName:03fb7e55-a065-427b-9404-45db03b4293e nodeName:}" failed. No retries permitted until 2026-08-04 00:03:20.405646324 +0000 UTC m=+66.286093856 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs") pod "network-metrics-daemon-kkl4r" (UID: "03fb7e55-a065-427b-9404-45db03b4293e") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 00:02:48.506294 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.506260 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:48.506553 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.506408 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 00:02:48.506553 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.506429 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 00:02:48.506553 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.506439 2551 projected.go:196] Error preparing data for projected volume kube-api-access-928pd for pod openshift-network-diagnostics/network-check-target-dnb4v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:48.506553 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:48.506486 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd podName:5e843f12-703e-48d6-9e00-a121b25098f3 nodeName:}" failed. No retries permitted until 2026-08-04 00:03:20.506473662 +0000 UTC m=+66.386921181 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-928pd" (UniqueName: "kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd") pod "network-check-target-dnb4v" (UID: "5e843f12-703e-48d6-9e00-a121b25098f3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 00:02:48.548145 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.548117 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:48.560970 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.560945 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6k6zs" Aug 04 00:02:48.606844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.606815 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:48.653079 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.653043 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:02:48.653870 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.653046 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:02:48.656710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.656052 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 00:02:48.656710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.656334 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 00:02:48.656710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.656338 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j4cq2\"" Aug 04 00:02:48.656710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.656505 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 00:02:48.656710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.656572 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l74qs\"" Aug 04 00:02:48.725922 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.725890 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6k6zs"] Aug 04 00:02:48.729407 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:48.729381 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc834e8_f024_4f19_8b8f_b84b2f9d1f39.slice/crio-bc722b146483c673a43653983a05843430f382dfd9a31133d10419578ab90c60 WatchSource:0}: Error finding container bc722b146483c673a43653983a05843430f382dfd9a31133d10419578ab90c60: Status 404 returned error can't find the container with id bc722b146483c673a43653983a05843430f382dfd9a31133d10419578ab90c60 Aug 04 00:02:48.785702 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.785664 2551 generic.go:356] "Generic (PLEG): container finished" podID="365c333a-8898-472b-8c70-21345d778a50" containerID="e11051de76eff7eb1f98f3ebc1154571d2d4bb1f886870d2a3c092d90d41e8e0" exitCode=0 Aug 04 00:02:48.785889 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.785747 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerDied","Data":"e11051de76eff7eb1f98f3ebc1154571d2d4bb1f886870d2a3c092d90d41e8e0"} Aug 04 00:02:48.787015 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.786965 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6k6zs" event={"ID":"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39","Type":"ContainerStarted","Data":"bc722b146483c673a43653983a05843430f382dfd9a31133d10419578ab90c60"} Aug 04 00:02:48.788231 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:48.788204 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sx2pp" event={"ID":"dba2ddf7-7e20-424b-b5a3-06389a500ab7","Type":"ContainerStarted","Data":"e9080c955027e49a1394560e7fc9a2a525d503b069a92a9fe74cb8ea6f5e01b4"} Aug 04 00:02:49.315370 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:49.315292 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:49.315370 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:49.315349 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:49.315668 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:49.315443 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 00:02:49.315668 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:49.315462 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 00:02:49.315668 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:49.315528 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls podName:2d3e5909-e3db-42e4-a285-a991937e28f5 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:51.31550839 +0000 UTC m=+37.195955909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls") pod "dns-default-5nk69" (UID: "2d3e5909-e3db-42e4-a285-a991937e28f5") : secret "dns-default-metrics-tls" not found Aug 04 00:02:49.315668 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:49.315466 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8584f9c6d5-jr94n: secret "image-registry-tls" not found Aug 04 00:02:49.315668 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:49.315606 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls podName:288611d8-927d-40ec-b1c9-5be0092a8f6e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:51.315585704 +0000 UTC m=+37.196033225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls") pod "image-registry-8584f9c6d5-jr94n" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e") : secret "image-registry-tls" not found Aug 04 00:02:49.795232 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:49.795195 2551 generic.go:356] "Generic (PLEG): container finished" podID="365c333a-8898-472b-8c70-21345d778a50" containerID="1eaa80c640a91416d0efae93f9050a960f86a7a3f2e575c3a92706f4d7532179" exitCode=0 Aug 04 00:02:49.795665 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:49.795270 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerDied","Data":"1eaa80c640a91416d0efae93f9050a960f86a7a3f2e575c3a92706f4d7532179"} Aug 04 00:02:49.872788 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:49.872714 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:49.972746 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:49.972687 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:50.800427 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:50.800395 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" event={"ID":"365c333a-8898-472b-8c70-21345d778a50","Type":"ContainerStarted","Data":"c2adb2b05dfb3a7bf52f4b3fc0014d1607c32e031ada089a765dc904efe877de"} Aug 04 00:02:50.823245 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:50.823192 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k5rzl" podStartSLOduration=9.003735475 podStartE2EDuration="36.823171659s" totalImagesPullingTime="27.819436184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:50.822466837 +0000 UTC m=+36.702914378" watchObservedRunningTime="2026-08-04 00:02:50.823171659 +0000 UTC m=+36.703619202" Aug 04 00:02:50.941028 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:50.940995 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:51.041307 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:51.041115 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:51.331419 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:51.331384 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:51.331582 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:51.331440 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:51.331582 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:51.331548 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 00:02:51.331582 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:51.331571 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8584f9c6d5-jr94n: secret "image-registry-tls" not found Aug 04 00:02:51.331582 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:51.331573 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 00:02:51.331768 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:51.331638 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls podName:2d3e5909-e3db-42e4-a285-a991937e28f5 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:55.331619974 +0000 UTC m=+41.212067495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls") pod "dns-default-5nk69" (UID: "2d3e5909-e3db-42e4-a285-a991937e28f5") : secret "dns-default-metrics-tls" not found Aug 04 00:02:51.331768 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:51.331657 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls podName:288611d8-927d-40ec-b1c9-5be0092a8f6e nodeName:}" failed. No retries permitted until 2026-08-04 00:02:55.331648301 +0000 UTC m=+41.212095821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls") pod "image-registry-8584f9c6d5-jr94n" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e") : secret "image-registry-tls" not found Aug 04 00:02:51.805059 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:51.805023 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sx2pp" event={"ID":"dba2ddf7-7e20-424b-b5a3-06389a500ab7","Type":"ContainerStarted","Data":"995035f33220a6c11fb3056dff37402f4a451250705e4c17f7fe7f5e02427313"} Aug 04 00:02:51.820709 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:51.820660 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sx2pp" podStartSLOduration=2.084696303 podStartE2EDuration="4.820641256s" totalImagesPullingTime="2.735944953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:51.819911814 +0000 UTC m=+37.700359370" watchObservedRunningTime="2026-08-04 00:02:51.820641256 +0000 UTC m=+37.701088798" Aug 04 00:02:52.523407 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:52.523369 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:52.597951 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:52.597916 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:52.658323 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:52.658287 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:53.137144 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.137097 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:53.235926 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.235896 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:53.802424 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.802387 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:53.810955 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.810922 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6k6zs" event={"ID":"6bc834e8-f024-4f19-8b8f-b84b2f9d1f39","Type":"ContainerStarted","Data":"43a49681901f9c358ac44ebb6e519015fec168c3e28ce3e22a38e0be43275f51"} Aug 04 00:02:53.826361 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.826314 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6k6zs" podStartSLOduration=35.340758517 podStartE2EDuration="39.826300243s" totalImagesPullingTime="4.485541726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:53.825672063 +0000 UTC m=+39.706119615" watchObservedRunningTime="2026-08-04 00:02:53.826300243 +0000 UTC m=+39.706747782" Aug 04 00:02:53.910065 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.910015 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:53.972094 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.972065 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6"] Aug 04 00:02:53.988401 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.988380 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6"] Aug 04 00:02:53.988536 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.988474 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" Aug 04 00:02:53.990860 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.990839 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Aug 04 00:02:53.990955 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.990879 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8zmkv\"" Aug 04 00:02:53.990955 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.990905 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Aug 04 00:02:53.991872 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:53.991843 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:54.066063 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.066034 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:54.094878 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.094857 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zpdc6"] Aug 04 00:02:54.118362 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.118342 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zpdc6"] Aug 04 00:02:54.118440 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.118428 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.120897 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.120878 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 00:02:54.120994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.120946 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qvwq5\"" Aug 04 00:02:54.121098 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.121084 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 00:02:54.121259 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.121244 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 00:02:54.121504 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.121487 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 00:02:54.139162 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.139137 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:54.154230 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.154208 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxr6p\" (UniqueName: \"kubernetes.io/projected/016682c0-bacf-4a7b-a6f8-76cc9e7eff4b-kube-api-access-kxr6p\") pod \"migrator-68bc745d87-mndq6\" (UID: \"016682c0-bacf-4a7b-a6f8-76cc9e7eff4b\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" Aug 04 00:02:54.254756 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.254663 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxr6p\" (UniqueName: \"kubernetes.io/projected/016682c0-bacf-4a7b-a6f8-76cc9e7eff4b-kube-api-access-kxr6p\") pod \"migrator-68bc745d87-mndq6\" (UID: \"016682c0-bacf-4a7b-a6f8-76cc9e7eff4b\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" Aug 04 00:02:54.254756 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.254708 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8bxh\" (UniqueName: \"kubernetes.io/projected/63919cf6-3390-422c-b39e-c7803a1654d6-kube-api-access-v8bxh\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.254946 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.254930 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63919cf6-3390-422c-b39e-c7803a1654d6-crio-socket\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.254992 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.254963 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.255040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.255006 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63919cf6-3390-422c-b39e-c7803a1654d6-data-volume\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.255085 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.255053 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.255689 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.255660 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:54.264099 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.264077 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxr6p\" (UniqueName: \"kubernetes.io/projected/016682c0-bacf-4a7b-a6f8-76cc9e7eff4b-kube-api-access-kxr6p\") pod \"migrator-68bc745d87-mndq6\" (UID: \"016682c0-bacf-4a7b-a6f8-76cc9e7eff4b\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" Aug 04 00:02:54.297021 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.296995 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" Aug 04 00:02:54.356070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.355681 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.356070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.355745 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63919cf6-3390-422c-b39e-c7803a1654d6-data-volume\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.356070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.355769 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.356070 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.355911 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 00:02:54.356070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.355919 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8bxh\" (UniqueName: \"kubernetes.io/projected/63919cf6-3390-422c-b39e-c7803a1654d6-kube-api-access-v8bxh\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.356070 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.355976 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:54.855957241 +0000 UTC m=+40.736404760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : secret "insights-runtime-extractor-tls" not found Aug 04 00:02:54.356070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.356029 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63919cf6-3390-422c-b39e-c7803a1654d6-crio-socket\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.356518 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.356199 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 00:02:54.356518 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.356256 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:54.856239428 +0000 UTC m=+40.736686963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : configmap "kube-rbac-proxy" not found Aug 04 00:02:54.356518 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.356252 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63919cf6-3390-422c-b39e-c7803a1654d6-crio-socket\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.356648 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.356572 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63919cf6-3390-422c-b39e-c7803a1654d6-data-volume\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.357053 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.357028 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:54.365085 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.365059 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8bxh\" (UniqueName: \"kubernetes.io/projected/63919cf6-3390-422c-b39e-c7803a1654d6-kube-api-access-v8bxh\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.428560 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.428533 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6"] Aug 04 00:02:54.431838 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:54.431814 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod016682c0_bacf_4a7b_a6f8_76cc9e7eff4b.slice/crio-e88e62a3b1681eadbbdf3b0e086fad212b3e1a782beedb306fca402c2ae83613 WatchSource:0}: Error finding container e88e62a3b1681eadbbdf3b0e086fad212b3e1a782beedb306fca402c2ae83613: Status 404 returned error can't find the container with id e88e62a3b1681eadbbdf3b0e086fad212b3e1a782beedb306fca402c2ae83613 Aug 04 00:02:54.620019 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.619930 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:54.679868 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.679834 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:54.814474 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.814443 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" event={"ID":"016682c0-bacf-4a7b-a6f8-76cc9e7eff4b","Type":"ContainerStarted","Data":"e88e62a3b1681eadbbdf3b0e086fad212b3e1a782beedb306fca402c2ae83613"} Aug 04 00:02:54.861004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.860975 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.861131 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:54.861016 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:54.861131 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.861085 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 00:02:54.861131 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.861127 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 00:02:54.861239 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.861154 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:55.8611346 +0000 UTC m=+41.741582132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : configmap "kube-rbac-proxy" not found Aug 04 00:02:54.861239 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:54.861176 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:55.861165012 +0000 UTC m=+41.741612550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : secret "insights-runtime-extractor-tls" not found Aug 04 00:02:55.364967 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:55.364934 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:02:55.365433 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:55.365029 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:02:55.365433 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.365098 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 00:02:55.365433 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.365162 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls podName:2d3e5909-e3db-42e4-a285-a991937e28f5 nodeName:}" failed. No retries permitted until 2026-08-04 00:03:03.365142831 +0000 UTC m=+49.245590369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls") pod "dns-default-5nk69" (UID: "2d3e5909-e3db-42e4-a285-a991937e28f5") : secret "dns-default-metrics-tls" not found Aug 04 00:02:55.365433 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.365189 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 00:02:55.365433 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.365205 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8584f9c6d5-jr94n: secret "image-registry-tls" not found Aug 04 00:02:55.365433 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.365253 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls podName:288611d8-927d-40ec-b1c9-5be0092a8f6e nodeName:}" failed. No retries permitted until 2026-08-04 00:03:03.365237539 +0000 UTC m=+49.245685072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls") pod "image-registry-8584f9c6d5-jr94n" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e") : secret "image-registry-tls" not found Aug 04 00:02:55.729628 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:55.729588 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:55.783445 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:55.783396 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:55.869185 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:55.869153 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:55.869329 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:55.869245 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:55.869329 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.869293 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 00:02:55.869329 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.869319 2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 00:02:55.869425 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.869361 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:57.869346048 +0000 UTC m=+43.749793566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : secret "insights-runtime-extractor-tls" not found Aug 04 00:02:55.869425 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:55.869375 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:02:57.869368429 +0000 UTC m=+43.749815946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : configmap "kube-rbac-proxy" not found Aug 04 00:02:56.201163 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.201136 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-76tgv"] Aug 04 00:02:56.232381 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.232354 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-76tgv"] Aug 04 00:02:56.232480 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.232417 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.235300 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.235279 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Aug 04 00:02:56.235300 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.235296 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Aug 04 00:02:56.235443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.235335 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Aug 04 00:02:56.236278 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.236262 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rz76j\"" Aug 04 00:02:56.236357 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.236337 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Aug 04 00:02:56.242140 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.242117 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:56.272164 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.272143 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/184b2602-6378-4016-93a4-be4a08229b46-signing-key\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.272249 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.272181 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h8mn\" (UniqueName: \"kubernetes.io/projected/184b2602-6378-4016-93a4-be4a08229b46-kube-api-access-7h8mn\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.272325 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.272308 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/184b2602-6378-4016-93a4-be4a08229b46-signing-cabundle\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.302581 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.302564 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b5tk9_315bfe51-8619-4ecf-9e28-a1cdb0b5eebf/dns-node-resolver/0.log" Aug 04 00:02:56.338731 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.338706 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:56.372935 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.372905 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/184b2602-6378-4016-93a4-be4a08229b46-signing-cabundle\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.373233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.372952 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/184b2602-6378-4016-93a4-be4a08229b46-signing-key\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.373233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.372989 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h8mn\" (UniqueName: \"kubernetes.io/projected/184b2602-6378-4016-93a4-be4a08229b46-kube-api-access-7h8mn\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.373628 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.373602 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/184b2602-6378-4016-93a4-be4a08229b46-signing-cabundle\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.375198 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.375182 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/184b2602-6378-4016-93a4-be4a08229b46-signing-key\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.381138 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.381115 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h8mn\" (UniqueName: \"kubernetes.io/projected/184b2602-6378-4016-93a4-be4a08229b46-kube-api-access-7h8mn\") pod \"service-ca-98c88fbb5-76tgv\" (UID: \"184b2602-6378-4016-93a4-be4a08229b46\") " pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.541355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.541284 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" Aug 04 00:02:56.697763 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.697732 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-76tgv"] Aug 04 00:02:56.700627 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:02:56.700600 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184b2602_6378_4016_93a4_be4a08229b46.slice/crio-4736596cce3607d970be1583f94c5d30693dca33b9baeaacc025871bba8778b2 WatchSource:0}: Error finding container 4736596cce3607d970be1583f94c5d30693dca33b9baeaacc025871bba8778b2: Status 404 returned error can't find the container with id 4736596cce3607d970be1583f94c5d30693dca33b9baeaacc025871bba8778b2 Aug 04 00:02:56.820097 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.820016 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" event={"ID":"184b2602-6378-4016-93a4-be4a08229b46","Type":"ContainerStarted","Data":"4736596cce3607d970be1583f94c5d30693dca33b9baeaacc025871bba8778b2"} Aug 04 00:02:56.821469 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.821444 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" event={"ID":"016682c0-bacf-4a7b-a6f8-76cc9e7eff4b","Type":"ContainerStarted","Data":"eb1adec89d4d6d17b168c5165a900b465d9839f1bf011cae71e75104a1227d3a"} Aug 04 00:02:56.821556 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.821486 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" event={"ID":"016682c0-bacf-4a7b-a6f8-76cc9e7eff4b","Type":"ContainerStarted","Data":"dbde608810b480b6a3045e94e36923927f922f86fcb40ef4b7884e73b80f963d"} Aug 04 00:02:56.832815 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.832778 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:56.838577 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:56.838481 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-mndq6" podStartSLOduration=2.114515029 podStartE2EDuration="3.8384659s" totalImagesPullingTime="1.723950871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:56.838155131 +0000 UTC m=+42.718602671" watchObservedRunningTime="2026-08-04 00:02:56.8384659 +0000 UTC m=+42.718913441" Aug 04 00:02:57.010791 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:57.010740 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:02:57.503403 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:57.503378 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pgjn4_f3d39dda-92c3-44bf-b400-ab782afb57c3/node-ca/0.log" Aug 04 00:02:57.884991 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:57.884960 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:57.885203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:57.885022 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:57.885203 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:57.885179 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 00:02:57.885325 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:02:57.885245 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:03:01.885225328 +0000 UTC m=+47.765672864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : secret "insights-runtime-extractor-tls" not found Aug 04 00:02:57.885765 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:57.885733 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63919cf6-3390-422c-b39e-c7803a1654d6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:02:58.503237 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:58.503188 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sx2pp_dba2ddf7-7e20-424b-b5a3-06389a500ab7/serve-healthcheck-canary/0.log" Aug 04 00:02:58.704231 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:58.704193 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-mndq6_016682c0-bacf-4a7b-a6f8-76cc9e7eff4b/migrator/0.log" Aug 04 00:02:58.903165 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:58.903127 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-mndq6_016682c0-bacf-4a7b-a6f8-76cc9e7eff4b/graceful-termination/0.log" Aug 04 00:02:59.830467 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:59.830435 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" event={"ID":"184b2602-6378-4016-93a4-be4a08229b46","Type":"ContainerStarted","Data":"f345b25022ff5bcb30c7033bc4b8e4451f35a0ee8ab407be7f10867e7e3cc06a"} Aug 04 00:02:59.848235 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:02:59.848186 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-76tgv" podStartSLOduration=1.54522303 podStartE2EDuration="3.848170831s" totalImagesPullingTime="2.302947801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:02:59.84729333 +0000 UTC m=+45.727740873" watchObservedRunningTime="2026-08-04 00:02:59.848170831 +0000 UTC m=+45.728618442" Aug 04 00:03:00.143071 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:00.143037 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:00.286772 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:00.286739 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:00.388242 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:00.388204 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:00.506721 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:00.506614 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:01.916063 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:01.916025 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:03:01.916458 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:01.916157 2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 00:03:01.916458 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:01.916210 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls podName:63919cf6-3390-422c-b39e-c7803a1654d6 nodeName:}" failed. No retries permitted until 2026-08-04 00:03:09.916195635 +0000 UTC m=+55.796643152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zpdc6" (UID: "63919cf6-3390-422c-b39e-c7803a1654d6") : secret "insights-runtime-extractor-tls" not found Aug 04 00:03:02.596085 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:02.596052 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:02.646043 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:02.646015 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:02.784614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:02.784575 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:02.981261 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:02.981220 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:03.427539 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:03.427510 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") pod \"image-registry-8584f9c6d5-jr94n\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:03:03.427674 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:03.427551 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:03:03.427674 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:03.427663 2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 00:03:03.427759 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:03.427681 2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8584f9c6d5-jr94n: secret "image-registry-tls" not found Aug 04 00:03:03.427759 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:03.427690 2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 00:03:03.427759 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:03.427738 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls podName:288611d8-927d-40ec-b1c9-5be0092a8f6e nodeName:}" failed. No retries permitted until 2026-08-04 00:03:19.427721872 +0000 UTC m=+65.308169395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls") pod "image-registry-8584f9c6d5-jr94n" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e") : secret "image-registry-tls" not found Aug 04 00:03:03.427759 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:03.427753 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls podName:2d3e5909-e3db-42e4-a285-a991937e28f5 nodeName:}" failed. No retries permitted until 2026-08-04 00:03:19.427747255 +0000 UTC m=+65.308194773 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls") pod "dns-default-5nk69" (UID: "2d3e5909-e3db-42e4-a285-a991937e28f5") : secret "dns-default-metrics-tls" not found Aug 04 00:03:03.801528 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:03.801433 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:03.988602 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:03.988564 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:04.179359 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:04.179329 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:04.379320 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:04.379288 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:04.580624 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:04.580529 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:04.644367 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:04.644332 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:04.781523 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:04.781487 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:04.833714 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:04.833634 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:04.906898 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:04.906873 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:05.005010 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:05.004978 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:05.058830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:05.058783 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:05.981805 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:05.981772 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:06.181250 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:06.181215 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:06.382786 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:06.382754 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:06.539198 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:06.539165 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:06.640176 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:06.640081 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:07.584721 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:07.584689 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:07.794402 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:07.794370 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:08.180416 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:08.180383 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:08.382070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:08.382038 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:08.505128 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:08.505045 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:08.786549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:08.786464 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:09.580616 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:09.580582 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:09.783813 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:09.783767 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:09.974981 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:09.974950 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:03:09.977232 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:09.977210 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63919cf6-3390-422c-b39e-c7803a1654d6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zpdc6\" (UID: \"63919cf6-3390-422c-b39e-c7803a1654d6\") " pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:03:09.981676 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:09.981648 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:10.026633 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:10.026609 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zpdc6" Aug 04 00:03:10.150406 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:10.150377 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zpdc6"] Aug 04 00:03:10.153604 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:10.153579 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63919cf6_3390_422c_b39e_c7803a1654d6.slice/crio-2b5e05e42a9a70cfbfec47f8a2e172baeb9f58982ce0ddd947332a3968c1989e WatchSource:0}: Error finding container 2b5e05e42a9a70cfbfec47f8a2e172baeb9f58982ce0ddd947332a3968c1989e: Status 404 returned error can't find the container with id 2b5e05e42a9a70cfbfec47f8a2e172baeb9f58982ce0ddd947332a3968c1989e Aug 04 00:03:10.410406 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:10.410371 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:10.785277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:10.785201 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:10.859254 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:10.859223 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zpdc6" event={"ID":"63919cf6-3390-422c-b39e-c7803a1654d6","Type":"ContainerStarted","Data":"edf43a3675303b05e9c1b992996c20bfbb0b64182da9874bb9db0514f3ed3398"} Aug 04 00:03:10.859403 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:10.859281 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zpdc6" event={"ID":"63919cf6-3390-422c-b39e-c7803a1654d6","Type":"ContainerStarted","Data":"2b5e05e42a9a70cfbfec47f8a2e172baeb9f58982ce0ddd947332a3968c1989e"} Aug 04 00:03:10.985467 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:10.985439 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:11.864156 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:11.864109 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zpdc6" event={"ID":"63919cf6-3390-422c-b39e-c7803a1654d6","Type":"ContainerStarted","Data":"72f08ad9d6f5b124a18a49ae7db7604cdae8826755aa70021577c01b56cdfa91"} Aug 04 00:03:13.710462 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:13.710390 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:13.855394 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:13.855343 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:13.876721 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:13.876678 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zpdc6" event={"ID":"63919cf6-3390-422c-b39e-c7803a1654d6","Type":"ContainerStarted","Data":"e0cd9ede3c18e6bd9e3cfd914a328aca63ac2b2d4ebde302c08b097eb5c5adae"} Aug 04 00:03:13.897305 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:13.897266 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zpdc6" podStartSLOduration=17.225827609 podStartE2EDuration="19.897253138s" totalImagesPullingTime="2.671425529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 00:02:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:13.896976221 +0000 UTC m=+59.777423769" watchObservedRunningTime="2026-08-04 00:03:13.897253138 +0000 UTC m=+59.777700677" Aug 04 00:03:14.096384 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.096306 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.296951 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.296894 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.398045 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.398008 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.445728 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.445704 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-cw62v"] Aug 04 00:03:14.448171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.448153 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.450644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.450623 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xd5sm\"" Aug 04 00:03:14.451590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.451571 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Aug 04 00:03:14.451693 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.451591 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Aug 04 00:03:14.465467 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.465446 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-cw62v"] Aug 04 00:03:14.492677 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.492656 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.507692 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.507668 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/397af7b7-a308-43bc-b985-6b61dbc61347-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-cw62v\" (UID: \"397af7b7-a308-43bc-b985-6b61dbc61347\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.507776 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.507706 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/397af7b7-a308-43bc-b985-6b61dbc61347-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-cw62v\" (UID: \"397af7b7-a308-43bc-b985-6b61dbc61347\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.598121 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.598087 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9"] Aug 04 00:03:14.600967 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.600951 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" Aug 04 00:03:14.604535 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.604513 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 04 00:03:14.608619 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.608596 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/397af7b7-a308-43bc-b985-6b61dbc61347-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-cw62v\" (UID: \"397af7b7-a308-43bc-b985-6b61dbc61347\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.608739 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.608720 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/397af7b7-a308-43bc-b985-6b61dbc61347-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-cw62v\" (UID: \"397af7b7-a308-43bc-b985-6b61dbc61347\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.609585 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.609562 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-spzpj\"" Aug 04 00:03:14.611724 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.611707 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Aug 04 00:03:14.612644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.612628 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Aug 04 00:03:14.619219 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.619201 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/397af7b7-a308-43bc-b985-6b61dbc61347-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-cw62v\" (UID: \"397af7b7-a308-43bc-b985-6b61dbc61347\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.621315 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.621298 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/397af7b7-a308-43bc-b985-6b61dbc61347-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-cw62v\" (UID: \"397af7b7-a308-43bc-b985-6b61dbc61347\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.626954 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.626934 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9"] Aug 04 00:03:14.630834 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.630815 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8584f9c6d5-jr94n"] Aug 04 00:03:14.631038 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:14.631017 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" podUID="288611d8-927d-40ec-b1c9-5be0092a8f6e" Aug 04 00:03:14.667253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.667190 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.670725 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.670704 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-r577z"] Aug 04 00:03:14.673993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.673979 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-r577z" Aug 04 00:03:14.676106 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.676090 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 00:03:14.676529 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.676511 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-x2x5x\"" Aug 04 00:03:14.676529 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.676523 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 00:03:14.684577 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.684549 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-r577z"] Aug 04 00:03:14.709635 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.709614 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rwx5\" (UniqueName: \"kubernetes.io/projected/88a5d2f3-3361-4999-8823-d802b9f67fad-kube-api-access-4rwx5\") pod \"downloads-77c974ffcf-r577z\" (UID: \"88a5d2f3-3361-4999-8823-d802b9f67fad\") " pod="openshift-console/downloads-77c974ffcf-r577z" Aug 04 00:03:14.709737 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.709643 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/32c81e49-2f55-4fc3-8cfe-1e41abd29d18-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-tz6c9\" (UID: \"32c81e49-2f55-4fc3-8cfe-1e41abd29d18\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" Aug 04 00:03:14.742163 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.742133 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.759024 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.759002 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xd5sm\"" Aug 04 00:03:14.765261 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.765240 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85ffd74c56-qkg67"] Aug 04 00:03:14.767317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.767302 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" Aug 04 00:03:14.767829 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.767816 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.781852 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.781828 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85ffd74c56-qkg67"] Aug 04 00:03:14.784098 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.784065 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ql2gr" Aug 04 00:03:14.810405 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810377 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rwx5\" (UniqueName: \"kubernetes.io/projected/88a5d2f3-3361-4999-8823-d802b9f67fad-kube-api-access-4rwx5\") pod \"downloads-77c974ffcf-r577z\" (UID: \"88a5d2f3-3361-4999-8823-d802b9f67fad\") " pod="openshift-console/downloads-77c974ffcf-r577z" Aug 04 00:03:14.810562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810413 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/32c81e49-2f55-4fc3-8cfe-1e41abd29d18-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-tz6c9\" (UID: \"32c81e49-2f55-4fc3-8cfe-1e41abd29d18\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" Aug 04 00:03:14.810562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810450 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5125ae2-f4a8-4d00-b77d-90af11295854-image-registry-private-configuration\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.810562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810466 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5125ae2-f4a8-4d00-b77d-90af11295854-trusted-ca\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.810562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810504 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5125ae2-f4a8-4d00-b77d-90af11295854-installation-pull-secrets\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.810562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810533 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5125ae2-f4a8-4d00-b77d-90af11295854-registry-certificates\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.810562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810550 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzk9b\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-kube-api-access-fzk9b\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.810900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810724 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-registry-tls\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.810900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810817 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5125ae2-f4a8-4d00-b77d-90af11295854-ca-trust-extracted\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.810900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.810845 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-bound-sa-token\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.812653 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.812628 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/32c81e49-2f55-4fc3-8cfe-1e41abd29d18-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-tz6c9\" (UID: \"32c81e49-2f55-4fc3-8cfe-1e41abd29d18\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" Aug 04 00:03:14.820700 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.820622 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rwx5\" (UniqueName: \"kubernetes.io/projected/88a5d2f3-3361-4999-8823-d802b9f67fad-kube-api-access-4rwx5\") pod \"downloads-77c974ffcf-r577z\" (UID: \"88a5d2f3-3361-4999-8823-d802b9f67fad\") " pod="openshift-console/downloads-77c974ffcf-r577z" Aug 04 00:03:14.835319 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.835280 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.880228 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.880203 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:03:14.887632 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.887611 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:03:14.896998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.896979 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-cw62v"] Aug 04 00:03:14.899743 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:14.899721 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397af7b7_a308_43bc_b985_6b61dbc61347.slice/crio-b491325202ef737c2bb41c9bbba0f9d89d3cf9f3b4d5536f38d4e60dc02983c1 WatchSource:0}: Error finding container b491325202ef737c2bb41c9bbba0f9d89d3cf9f3b4d5536f38d4e60dc02983c1: Status 404 returned error can't find the container with id b491325202ef737c2bb41c9bbba0f9d89d3cf9f3b4d5536f38d4e60dc02983c1 Aug 04 00:03:14.910949 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.910931 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" Aug 04 00:03:14.911126 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911106 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-image-registry-private-configuration\") pod \"288611d8-927d-40ec-b1c9-5be0092a8f6e\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " Aug 04 00:03:14.911173 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911161 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-bound-sa-token\") pod \"288611d8-927d-40ec-b1c9-5be0092a8f6e\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " Aug 04 00:03:14.911208 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911194 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpvj4\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-kube-api-access-tpvj4\") pod \"288611d8-927d-40ec-b1c9-5be0092a8f6e\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " Aug 04 00:03:14.911240 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911232 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/288611d8-927d-40ec-b1c9-5be0092a8f6e-ca-trust-extracted\") pod \"288611d8-927d-40ec-b1c9-5be0092a8f6e\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " Aug 04 00:03:14.911289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911257 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-trusted-ca\") pod \"288611d8-927d-40ec-b1c9-5be0092a8f6e\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " Aug 04 00:03:14.911341 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911296 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-certificates\") pod \"288611d8-927d-40ec-b1c9-5be0092a8f6e\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " Aug 04 00:03:14.911341 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911324 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-installation-pull-secrets\") pod \"288611d8-927d-40ec-b1c9-5be0092a8f6e\" (UID: \"288611d8-927d-40ec-b1c9-5be0092a8f6e\") " Aug 04 00:03:14.911463 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911419 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-registry-tls\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911547 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911461 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5125ae2-f4a8-4d00-b77d-90af11295854-ca-trust-extracted\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911547 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911499 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-bound-sa-token\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911688 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911576 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5125ae2-f4a8-4d00-b77d-90af11295854-image-registry-private-configuration\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911688 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911602 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5125ae2-f4a8-4d00-b77d-90af11295854-trusted-ca\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911688 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911643 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5125ae2-f4a8-4d00-b77d-90af11295854-installation-pull-secrets\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911859 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911708 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5125ae2-f4a8-4d00-b77d-90af11295854-registry-certificates\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911859 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911738 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzk9b\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-kube-api-access-fzk9b\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.911959 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.911849 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/288611d8-927d-40ec-b1c9-5be0092a8f6e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "288611d8-927d-40ec-b1c9-5be0092a8f6e" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:03:14.912271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.912252 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "288611d8-927d-40ec-b1c9-5be0092a8f6e" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:03:14.913602 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.913335 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5125ae2-f4a8-4d00-b77d-90af11295854-trusted-ca\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.913930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.913653 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "288611d8-927d-40ec-b1c9-5be0092a8f6e" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:03:14.913930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.913685 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5125ae2-f4a8-4d00-b77d-90af11295854-ca-trust-extracted\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.913930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.913704 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "288611d8-927d-40ec-b1c9-5be0092a8f6e" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:03:14.913930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.913750 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "288611d8-927d-40ec-b1c9-5be0092a8f6e" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:03:14.914169 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.914145 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "288611d8-927d-40ec-b1c9-5be0092a8f6e" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:03:14.914324 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.914297 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-kube-api-access-tpvj4" (OuterVolumeSpecName: "kube-api-access-tpvj4") pod "288611d8-927d-40ec-b1c9-5be0092a8f6e" (UID: "288611d8-927d-40ec-b1c9-5be0092a8f6e"). InnerVolumeSpecName "kube-api-access-tpvj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:03:14.914744 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.914715 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5125ae2-f4a8-4d00-b77d-90af11295854-registry-certificates\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.915110 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.915093 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-registry-tls\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.915983 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.915962 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5125ae2-f4a8-4d00-b77d-90af11295854-installation-pull-secrets\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.916208 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.916190 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5125ae2-f4a8-4d00-b77d-90af11295854-image-registry-private-configuration\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.932176 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.932149 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-bound-sa-token\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.933022 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.933000 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzk9b\" (UniqueName: \"kubernetes.io/projected/d5125ae2-f4a8-4d00-b77d-90af11295854-kube-api-access-fzk9b\") pod \"image-registry-85ffd74c56-qkg67\" (UID: \"d5125ae2-f4a8-4d00-b77d-90af11295854\") " pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:14.946866 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.946833 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:14.987103 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:14.987066 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-r577z" Aug 04 00:03:15.012505 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.012479 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpvj4\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-kube-api-access-tpvj4\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:15.012505 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.012501 2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/288611d8-927d-40ec-b1c9-5be0092a8f6e-ca-trust-extracted\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:15.012505 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.012512 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-trusted-ca\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:15.012734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.012521 2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-certificates\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:15.012734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.012531 2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-installation-pull-secrets\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:15.012734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.012540 2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/288611d8-927d-40ec-b1c9-5be0092a8f6e-image-registry-private-configuration\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:15.012734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.012548 2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-bound-sa-token\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:15.030471 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.030446 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9"] Aug 04 00:03:15.033204 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:15.033180 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c81e49_2f55_4fc3_8cfe_1e41abd29d18.slice/crio-6156108c267906ab939c1227309fd8f26a05e189fc99320f6abc650faae0cc2e WatchSource:0}: Error finding container 6156108c267906ab939c1227309fd8f26a05e189fc99320f6abc650faae0cc2e: Status 404 returned error can't find the container with id 6156108c267906ab939c1227309fd8f26a05e189fc99320f6abc650faae0cc2e Aug 04 00:03:15.033914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.033884 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.089512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.089489 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dptb9\"" Aug 04 00:03:15.097840 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.097817 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:15.107366 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.107342 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-r577z"] Aug 04 00:03:15.109984 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:15.109957 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a5d2f3_3361_4999_8823_d802b9f67fad.slice/crio-e2ffff59e12ef15a7f8531b0e4d87dca94c17defbecbe84cac4b937eefd70572 WatchSource:0}: Error finding container e2ffff59e12ef15a7f8531b0e4d87dca94c17defbecbe84cac4b937eefd70572: Status 404 returned error can't find the container with id e2ffff59e12ef15a7f8531b0e4d87dca94c17defbecbe84cac4b937eefd70572 Aug 04 00:03:15.206934 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.206905 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.236017 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.235986 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85ffd74c56-qkg67"] Aug 04 00:03:15.238872 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:15.238810 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5125ae2_f4a8_4d00_b77d_90af11295854.slice/crio-0edc6b3b6c3f5bf156bed3c9b1db5492d0f2ef19c7541990e25473740466a0ac WatchSource:0}: Error finding container 0edc6b3b6c3f5bf156bed3c9b1db5492d0f2ef19c7541990e25473740466a0ac: Status 404 returned error can't find the container with id 0edc6b3b6c3f5bf156bed3c9b1db5492d0f2ef19c7541990e25473740466a0ac Aug 04 00:03:15.345932 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.345900 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.407532 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.407497 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.539937 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.539778 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.597121 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.597079 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.773850 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.773816 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.890032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.889972 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" event={"ID":"397af7b7-a308-43bc-b985-6b61dbc61347","Type":"ContainerStarted","Data":"b491325202ef737c2bb41c9bbba0f9d89d3cf9f3b4d5536f38d4e60dc02983c1"} Aug 04 00:03:15.891335 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.891297 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-r577z" event={"ID":"88a5d2f3-3361-4999-8823-d802b9f67fad","Type":"ContainerStarted","Data":"e2ffff59e12ef15a7f8531b0e4d87dca94c17defbecbe84cac4b937eefd70572"} Aug 04 00:03:15.892579 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.892553 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" event={"ID":"32c81e49-2f55-4fc3-8cfe-1e41abd29d18","Type":"ContainerStarted","Data":"6156108c267906ab939c1227309fd8f26a05e189fc99320f6abc650faae0cc2e"} Aug 04 00:03:15.894538 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.894517 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8584f9c6d5-jr94n" Aug 04 00:03:15.894645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.894550 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" event={"ID":"d5125ae2-f4a8-4d00-b77d-90af11295854","Type":"ContainerStarted","Data":"291dee2de8b5e6ca94cc1da3fabf3e2ba1d3e27fc2863b7fd399737b48d8f248"} Aug 04 00:03:15.894645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.894597 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" event={"ID":"d5125ae2-f4a8-4d00-b77d-90af11295854","Type":"ContainerStarted","Data":"0edc6b3b6c3f5bf156bed3c9b1db5492d0f2ef19c7541990e25473740466a0ac"} Aug 04 00:03:15.917603 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.917549 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" podStartSLOduration=1.9175328729999999 podStartE2EDuration="1.917532873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:15.915998535 +0000 UTC m=+61.796446079" watchObservedRunningTime="2026-08-04 00:03:15.917532873 +0000 UTC m=+61.797980410" Aug 04 00:03:15.950532 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.950483 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:15.951550 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.951523 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8584f9c6d5-jr94n"] Aug 04 00:03:15.953507 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:15.953484 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8584f9c6d5-jr94n"] Aug 04 00:03:16.001893 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.001860 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:16.023030 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.023000 2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/288611d8-927d-40ec-b1c9-5be0092a8f6e-registry-tls\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:03:16.385504 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.385469 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:16.658264 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.658194 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="288611d8-927d-40ec-b1c9-5be0092a8f6e" path="/var/lib/kubelet/pods/288611d8-927d-40ec-b1c9-5be0092a8f6e/volumes" Aug 04 00:03:16.865093 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.865043 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:16.900691 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.900653 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" event={"ID":"32c81e49-2f55-4fc3-8cfe-1e41abd29d18","Type":"ContainerStarted","Data":"742fead115cf7720f36679fd3fcd9e077319a002a0d50e4f9435d26145440dc7"} Aug 04 00:03:16.900872 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.900830 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" Aug 04 00:03:16.902593 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.902564 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" event={"ID":"397af7b7-a308-43bc-b985-6b61dbc61347","Type":"ContainerStarted","Data":"a013c7c76ccc9768a954e4ad36d7452c8818ce5e807bd2fcc865c659312a0a66"} Aug 04 00:03:16.902828 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.902788 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:16.906855 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.906836 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" Aug 04 00:03:16.915705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.915621 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-tz6c9" podStartSLOduration=1.426562118 podStartE2EDuration="2.915604884s" totalImagesPullingTime="1.489042766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:16.91489769 +0000 UTC m=+62.795345235" watchObservedRunningTime="2026-08-04 00:03:16.915604884 +0000 UTC m=+62.796052425" Aug 04 00:03:16.928275 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:16.928233 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-cw62v" podStartSLOduration=1.306962742 podStartE2EDuration="2.928218537s" totalImagesPullingTime="1.621255795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:16.928037972 +0000 UTC m=+62.808485524" watchObservedRunningTime="2026-08-04 00:03:16.928218537 +0000 UTC m=+62.808666076" Aug 04 00:03:17.138822 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:17.138766 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:17.240170 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:17.240078 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:17.338748 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:17.338709 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:17.880366 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:17.880330 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:17.947259 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:17.947179 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:18.001687 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:18.001639 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:18.245499 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:18.245420 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:19.452505 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:19.452474 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:03:19.455149 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:19.455116 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d3e5909-e3db-42e4-a285-a991937e28f5-metrics-tls\") pod \"dns-default-5nk69\" (UID: \"2d3e5909-e3db-42e4-a285-a991937e28f5\") " pod="openshift-dns/dns-default-5nk69" Aug 04 00:03:19.645995 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:19.645966 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9x2gl\"" Aug 04 00:03:19.654219 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:19.654196 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5nk69" Aug 04 00:03:19.834985 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:19.834932 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5nk69"] Aug 04 00:03:19.838021 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:19.837992 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3e5909_e3db_42e4_a285_a991937e28f5.slice/crio-213bec30f2c2c1995e08765644f049427ce645534baed1b2cb754901e3e7b8ab WatchSource:0}: Error finding container 213bec30f2c2c1995e08765644f049427ce645534baed1b2cb754901e3e7b8ab: Status 404 returned error can't find the container with id 213bec30f2c2c1995e08765644f049427ce645534baed1b2cb754901e3e7b8ab Aug 04 00:03:19.915408 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:19.915360 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5nk69" event={"ID":"2d3e5909-e3db-42e4-a285-a991937e28f5","Type":"ContainerStarted","Data":"213bec30f2c2c1995e08765644f049427ce645534baed1b2cb754901e3e7b8ab"} Aug 04 00:03:20.064267 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.064170 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:20.132578 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.132520 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:20.460725 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.460687 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:03:20.463502 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.463288 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 00:03:20.467707 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.467681 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:20.474844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.474816 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fb7e55-a065-427b-9404-45db03b4293e-metrics-certs\") pod \"network-metrics-daemon-kkl4r\" (UID: \"03fb7e55-a065-427b-9404-45db03b4293e\") " pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:03:20.561481 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.561396 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:03:20.564180 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.564153 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 00:03:20.575193 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.575028 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 00:03:20.585890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.585863 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-928pd\" (UniqueName: \"kubernetes.io/projected/5e843f12-703e-48d6-9e00-a121b25098f3-kube-api-access-928pd\") pod \"network-check-target-dnb4v\" (UID: \"5e843f12-703e-48d6-9e00-a121b25098f3\") " pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:03:20.770193 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.769858 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j4cq2\"" Aug 04 00:03:20.774479 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.774455 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l74qs\"" Aug 04 00:03:20.777559 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.777525 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kkl4r" Aug 04 00:03:20.782777 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.782746 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:03:20.961723 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.961632 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kkl4r"] Aug 04 00:03:20.978551 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:20.978521 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dnb4v"] Aug 04 00:03:21.495375 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:21.495340 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fb7e55_a065_427b_9404_45db03b4293e.slice/crio-cbfa461b13166233b25a013952464d6eb66c22bba7bc88c14ee3a77dc21d67c0 WatchSource:0}: Error finding container cbfa461b13166233b25a013952464d6eb66c22bba7bc88c14ee3a77dc21d67c0: Status 404 returned error can't find the container with id cbfa461b13166233b25a013952464d6eb66c22bba7bc88c14ee3a77dc21d67c0 Aug 04 00:03:21.496048 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:21.496020 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e843f12_703e_48d6_9e00_a121b25098f3.slice/crio-fa017e9a645df398ae17673a0342132d98ca496fd9bc7579f6c0a043ed8d3f74 WatchSource:0}: Error finding container fa017e9a645df398ae17673a0342132d98ca496fd9bc7579f6c0a043ed8d3f74: Status 404 returned error can't find the container with id fa017e9a645df398ae17673a0342132d98ca496fd9bc7579f6c0a043ed8d3f74 Aug 04 00:03:21.924051 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:21.924009 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dnb4v" event={"ID":"5e843f12-703e-48d6-9e00-a121b25098f3","Type":"ContainerStarted","Data":"fa017e9a645df398ae17673a0342132d98ca496fd9bc7579f6c0a043ed8d3f74"} Aug 04 00:03:21.925141 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:21.925111 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kkl4r" event={"ID":"03fb7e55-a065-427b-9404-45db03b4293e","Type":"ContainerStarted","Data":"cbfa461b13166233b25a013952464d6eb66c22bba7bc88c14ee3a77dc21d67c0"} Aug 04 00:03:21.927152 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:21.927125 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5nk69" event={"ID":"2d3e5909-e3db-42e4-a285-a991937e28f5","Type":"ContainerStarted","Data":"cb74e87a077091b1c5e7d382630f60d30067baea21a644328af1b2c05e310d3d"} Aug 04 00:03:21.927267 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:21.927169 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5nk69" event={"ID":"2d3e5909-e3db-42e4-a285-a991937e28f5","Type":"ContainerStarted","Data":"a4364c757751e2ccbf6133e30f34b0dea6baebb94ed7c6cc1b2618c19376b719"} Aug 04 00:03:21.927267 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:21.927203 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5nk69" Aug 04 00:03:21.946957 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:21.946918 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5nk69" podStartSLOduration=33.243457637 podStartE2EDuration="34.946903458s" totalImagesPullingTime="1.703445821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:21.946250332 +0000 UTC m=+67.826697872" watchObservedRunningTime="2026-08-04 00:03:21.946903458 +0000 UTC m=+67.827350998" Aug 04 00:03:22.038211 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.038166 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.126067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.126029 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.228652 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.228485 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.282219 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.280929 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vzq8t"] Aug 04 00:03:22.294963 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.294926 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.298319 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.298296 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 00:03:22.298476 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.298455 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 00:03:22.298540 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.298485 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 00:03:22.298597 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.298417 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 00:03:22.298647 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.298445 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 00:03:22.298695 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.298372 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 00:03:22.298768 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.298305 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zm4p5\"" Aug 04 00:03:22.315546 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.315394 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.376828 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-sys\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.376873 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.376915 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhvw\" (UniqueName: \"kubernetes.io/projected/14616f46-049b-47c2-b8f3-8c65ae39308d-kube-api-access-nzhvw\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.376945 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-root\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.377045 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-wtmp\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.377072 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14616f46-049b-47c2-b8f3-8c65ae39308d-metrics-client-ca\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.377121 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-textfile\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.377148 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-tls\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.377758 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.377179 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-accelerators-collector-config\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477615 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477567 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-wtmp\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477615 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477615 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14616f46-049b-47c2-b8f3-8c65ae39308d-metrics-client-ca\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477641 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-textfile\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477668 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-tls\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477698 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-accelerators-collector-config\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477726 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-sys\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477751 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477790 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhvw\" (UniqueName: \"kubernetes.io/projected/14616f46-049b-47c2-b8f3-8c65ae39308d-kube-api-access-nzhvw\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477842 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-root\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.477947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.477923 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-root\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.478362 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.478053 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-wtmp\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.478735 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.478675 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14616f46-049b-47c2-b8f3-8c65ae39308d-metrics-client-ca\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.478980 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.478960 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-textfile\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.479087 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:22.479072 2551 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 04 00:03:22.479158 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:03:22.479148 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-tls podName:14616f46-049b-47c2-b8f3-8c65ae39308d nodeName:}" failed. No retries permitted until 2026-08-04 00:03:22.979129207 +0000 UTC m=+68.859576731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-tls") pod "node-exporter-vzq8t" (UID: "14616f46-049b-47c2-b8f3-8c65ae39308d") : secret "node-exporter-tls" not found Aug 04 00:03:22.480143 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.479830 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-accelerators-collector-config\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.480143 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.479924 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14616f46-049b-47c2-b8f3-8c65ae39308d-sys\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.482568 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.482544 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.492353 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.492311 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhvw\" (UniqueName: \"kubernetes.io/projected/14616f46-049b-47c2-b8f3-8c65ae39308d-kube-api-access-nzhvw\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.539332 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.539178 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.680735 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.680642 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.769093 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.769001 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.831590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.831554 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:22.981308 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.981276 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-tls\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:22.984344 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:22.984314 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14616f46-049b-47c2-b8f3-8c65ae39308d-node-exporter-tls\") pod \"node-exporter-vzq8t\" (UID: \"14616f46-049b-47c2-b8f3-8c65ae39308d\") " pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:23.210659 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:23.210620 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vzq8t" Aug 04 00:03:23.250575 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:23.250495 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:23.328694 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:23.328660 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:23.450525 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:23.450487 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:23.762582 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:23.762547 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:23.937787 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:23.937751 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vzq8t" event={"ID":"14616f46-049b-47c2-b8f3-8c65ae39308d","Type":"ContainerStarted","Data":"94d581d1727a56ae5fd6f0f17fc13b47ecdd9389d61aefb0a2b932d1fe59e6d4"} Aug 04 00:03:25.228844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:25.228672 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:25.301708 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:25.301623 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:25.443550 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:25.443513 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:25.580531 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:25.580439 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:25.685717 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:25.685685 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:25.776410 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:25.776376 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:26.687224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.687189 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-db9668795-zn7bm"] Aug 04 00:03:26.691640 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.691611 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.696282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.696036 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4slj5\"" Aug 04 00:03:26.696282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.696052 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 00:03:26.696282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.696085 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Aug 04 00:03:26.696282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.696097 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Aug 04 00:03:26.696282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.696040 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Aug 04 00:03:26.696282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.696036 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5ahr1l3dljfrl\"" Aug 04 00:03:26.700624 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.700268 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db9668795-zn7bm"] Aug 04 00:03:26.820854 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.820822 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dc018cb-9822-44db-95af-a010165f4432-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.821038 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.820863 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6dc018cb-9822-44db-95af-a010165f4432-audit-log\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.821038 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.820884 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-client-ca-bundle\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.821038 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.820919 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-secret-metrics-server-tls\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.821038 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.820967 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6dc018cb-9822-44db-95af-a010165f4432-metrics-server-audit-profiles\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.821038 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.821035 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m955k\" (UniqueName: \"kubernetes.io/projected/6dc018cb-9822-44db-95af-a010165f4432-kube-api-access-m955k\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.821275 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.821079 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-secret-metrics-server-client-certs\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922278 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922243 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dc018cb-9822-44db-95af-a010165f4432-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922289 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6dc018cb-9822-44db-95af-a010165f4432-audit-log\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922311 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-client-ca-bundle\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922337 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-secret-metrics-server-tls\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922362 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6dc018cb-9822-44db-95af-a010165f4432-metrics-server-audit-profiles\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922445 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m955k\" (UniqueName: \"kubernetes.io/projected/6dc018cb-9822-44db-95af-a010165f4432-kube-api-access-m955k\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922728 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922524 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-secret-metrics-server-client-certs\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.922790 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.922738 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6dc018cb-9822-44db-95af-a010165f4432-audit-log\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.923100 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.923074 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dc018cb-9822-44db-95af-a010165f4432-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.923492 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.923473 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6dc018cb-9822-44db-95af-a010165f4432-metrics-server-audit-profiles\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.925217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.925195 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-secret-metrics-server-client-certs\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.925341 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.925280 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-client-ca-bundle\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.925692 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.925676 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6dc018cb-9822-44db-95af-a010165f4432-secret-metrics-server-tls\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:26.930506 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:26.930485 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m955k\" (UniqueName: \"kubernetes.io/projected/6dc018cb-9822-44db-95af-a010165f4432-kube-api-access-m955k\") pod \"metrics-server-db9668795-zn7bm\" (UID: \"6dc018cb-9822-44db-95af-a010165f4432\") " pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:27.003111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.003026 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:27.041240 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.041190 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh"] Aug 04 00:03:27.041436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.041266 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:27.044924 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.044903 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.047386 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.047363 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Aug 04 00:03:27.047498 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.047485 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hrzz5\"" Aug 04 00:03:27.053451 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.053397 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh"] Aug 04 00:03:27.094069 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.094032 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:27.125055 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.125023 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8f5c806-cd10-4754-b0ee-626b79a38ff1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-qqqfh\" (UID: \"a8f5c806-cd10-4754-b0ee-626b79a38ff1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.125217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.125063 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmrz4\" (UniqueName: \"kubernetes.io/projected/a8f5c806-cd10-4754-b0ee-626b79a38ff1-kube-api-access-zmrz4\") pod \"monitoring-plugin-6dffbbd67-qqqfh\" (UID: \"a8f5c806-cd10-4754-b0ee-626b79a38ff1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.225868 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.225828 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8f5c806-cd10-4754-b0ee-626b79a38ff1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-qqqfh\" (UID: \"a8f5c806-cd10-4754-b0ee-626b79a38ff1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.226043 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.225877 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmrz4\" (UniqueName: \"kubernetes.io/projected/a8f5c806-cd10-4754-b0ee-626b79a38ff1-kube-api-access-zmrz4\") pod \"monitoring-plugin-6dffbbd67-qqqfh\" (UID: \"a8f5c806-cd10-4754-b0ee-626b79a38ff1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.228602 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.228574 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8f5c806-cd10-4754-b0ee-626b79a38ff1-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-qqqfh\" (UID: \"a8f5c806-cd10-4754-b0ee-626b79a38ff1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.236547 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.236528 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmrz4\" (UniqueName: \"kubernetes.io/projected/a8f5c806-cd10-4754-b0ee-626b79a38ff1-kube-api-access-zmrz4\") pod \"monitoring-plugin-6dffbbd67-qqqfh\" (UID: \"a8f5c806-cd10-4754-b0ee-626b79a38ff1\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.356472 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.356385 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:27.706809 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.706726 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:27.808439 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.808384 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:27.857679 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:27.857640 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:28.557962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:28.557918 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:29.587686 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:29.587637 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:29.639199 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:29.639164 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:31.929193 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:31.929155 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:31.933891 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:31.933866 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5nk69" Aug 04 00:03:32.031047 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.031011 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:32.720072 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.720046 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-db9668795-zn7bm"] Aug 04 00:03:32.723225 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:32.723193 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc018cb_9822_44db_95af_a010165f4432.slice/crio-5b2612226a4122dddcc1f09cbbf41381cd3dbdaa3eba0fa6036a8a0c3fc21b1b WatchSource:0}: Error finding container 5b2612226a4122dddcc1f09cbbf41381cd3dbdaa3eba0fa6036a8a0c3fc21b1b: Status 404 returned error can't find the container with id 5b2612226a4122dddcc1f09cbbf41381cd3dbdaa3eba0fa6036a8a0c3fc21b1b Aug 04 00:03:32.735928 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.735906 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh"] Aug 04 00:03:32.738271 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:03:32.738245 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f5c806_cd10_4754_b0ee_626b79a38ff1.slice/crio-d4855014ee8435dacec196d045163aa88e6bf52f15fdc82b399aee858f027445 WatchSource:0}: Error finding container d4855014ee8435dacec196d045163aa88e6bf52f15fdc82b399aee858f027445: Status 404 returned error can't find the container with id d4855014ee8435dacec196d045163aa88e6bf52f15fdc82b399aee858f027445 Aug 04 00:03:32.974614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.974573 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dnb4v" event={"ID":"5e843f12-703e-48d6-9e00-a121b25098f3","Type":"ContainerStarted","Data":"c939f4a079b5242029fb61e680b55e2c8797c9545a4ee6a681d21f2fce99945f"} Aug 04 00:03:32.975089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.974647 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:03:32.976841 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.976643 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kkl4r" event={"ID":"03fb7e55-a065-427b-9404-45db03b4293e","Type":"ContainerStarted","Data":"d745d7c270d858250c26a46e0016a579b93bb73e8613fe1936a66e30de1f8b29"} Aug 04 00:03:32.976841 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.976709 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kkl4r" event={"ID":"03fb7e55-a065-427b-9404-45db03b4293e","Type":"ContainerStarted","Data":"e815f46ab1eba756f9bd6535005e49c5d2cfc3fcff0a741e1629368ee9136d14"} Aug 04 00:03:32.978229 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.978163 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" event={"ID":"a8f5c806-cd10-4754-b0ee-626b79a38ff1","Type":"ContainerStarted","Data":"d4855014ee8435dacec196d045163aa88e6bf52f15fdc82b399aee858f027445"} Aug 04 00:03:32.980059 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.980035 2551 generic.go:356] "Generic (PLEG): container finished" podID="14616f46-049b-47c2-b8f3-8c65ae39308d" containerID="0eec2f4fcf4961becaed594058c9e78799c263b5eb473cd6e63c4ffb6214b9f5" exitCode=0 Aug 04 00:03:32.980148 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.980114 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vzq8t" event={"ID":"14616f46-049b-47c2-b8f3-8c65ae39308d","Type":"ContainerDied","Data":"0eec2f4fcf4961becaed594058c9e78799c263b5eb473cd6e63c4ffb6214b9f5"} Aug 04 00:03:32.981522 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.981498 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" event={"ID":"6dc018cb-9822-44db-95af-a010165f4432","Type":"ContainerStarted","Data":"5b2612226a4122dddcc1f09cbbf41381cd3dbdaa3eba0fa6036a8a0c3fc21b1b"} Aug 04 00:03:32.983114 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.983093 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-r577z" event={"ID":"88a5d2f3-3361-4999-8823-d802b9f67fad","Type":"ContainerStarted","Data":"1dd27b7569b665a5c6405dc15825e96a5905dda474507d9077d5f120a9a25619"} Aug 04 00:03:32.983398 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.983382 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-r577z" Aug 04 00:03:32.991705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:32.991657 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dnb4v" podStartSLOduration=67.939300694 podStartE2EDuration="1m18.991644423s" totalImagesPullingTime="11.052343729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:32.991272575 +0000 UTC m=+78.871720115" watchObservedRunningTime="2026-08-04 00:03:32.991644423 +0000 UTC m=+78.872091942" Aug 04 00:03:33.003076 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.003046 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-r577z" Aug 04 00:03:33.026890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.026839 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kkl4r" podStartSLOduration=67.980598997 podStartE2EDuration="1m19.026820329s" totalImagesPullingTime="11.046221332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:33.025473637 +0000 UTC m=+78.905921181" watchObservedRunningTime="2026-08-04 00:03:33.026820329 +0000 UTC m=+78.907267862" Aug 04 00:03:33.040439 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.040372 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-r577z" podStartSLOduration=1.475619836 podStartE2EDuration="19.040359249s" totalImagesPullingTime="17.564739413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:33.040296037 +0000 UTC m=+78.920743579" watchObservedRunningTime="2026-08-04 00:03:33.040359249 +0000 UTC m=+78.920806790" Aug 04 00:03:33.125074 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.125037 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:33.178103 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.178057 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:33.651508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.651465 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:33.754900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.754831 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:33.810356 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.810219 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:33.869230 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.869196 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:33.992603 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.992558 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vzq8t" event={"ID":"14616f46-049b-47c2-b8f3-8c65ae39308d","Type":"ContainerStarted","Data":"dd19d5680c1cdf010f2ea3acecc7cd1ce46af7b26680679d9d70f4fff9d2a7a8"} Aug 04 00:03:33.993199 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:33.992630 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vzq8t" event={"ID":"14616f46-049b-47c2-b8f3-8c65ae39308d","Type":"ContainerStarted","Data":"bbf4b1d22dca80391e5ab2683e282a34b9d085b3ce479a648facb0320475bf3b"} Aug 04 00:03:34.015299 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:34.015077 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vzq8t" podStartSLOduration=2.624741423 podStartE2EDuration="12.01505527s" totalImagesPullingTime="9.31202353s" totalInitContainerRuntime="78.290317ms" isStatefulPod=true podCreationTimestamp="2026-08-04 00:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:34.014245086 +0000 UTC m=+79.894692628" watchObservedRunningTime="2026-08-04 00:03:34.01505527 +0000 UTC m=+79.895502813" Aug 04 00:03:34.875555 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:34.875514 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:34.932516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:34.932231 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:35.005007 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:35.004924 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" event={"ID":"6dc018cb-9822-44db-95af-a010165f4432","Type":"ContainerStarted","Data":"bb8410abccac2b72c951c926452f6a9eb0830ce3aacb277e02dea552af980188"} Aug 04 00:03:35.025850 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:35.025354 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" podStartSLOduration=7.072443126 podStartE2EDuration="9.025332684s" totalImagesPullingTime="1.952889558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:35.022588416 +0000 UTC m=+80.903035957" watchObservedRunningTime="2026-08-04 00:03:35.025332684 +0000 UTC m=+80.905780232" Aug 04 00:03:35.105700 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:35.105656 2551 patch_prober.go:28] interesting pod/image-registry-85ffd74c56-qkg67 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 00:03:35.105885 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:35.105730 2551 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" podUID="d5125ae2-f4a8-4d00-b77d-90af11295854" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 00:03:35.828767 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:35.828729 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:35.881325 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:35.881284 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:36.770695 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:36.770659 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:37.014384 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:37.014256 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" event={"ID":"a8f5c806-cd10-4754-b0ee-626b79a38ff1","Type":"ContainerStarted","Data":"163569c3c613dbaa2618a1550d4135cc923a0f89c05ff932100ca4bc280542b8"} Aug 04 00:03:37.014539 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:37.014484 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:37.020628 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:37.020604 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" Aug 04 00:03:37.035324 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:37.035246 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-qqqfh" podStartSLOduration=6.718917875 podStartE2EDuration="10.03522874s" totalImagesPullingTime="3.316310865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:03:37.033331075 +0000 UTC m=+82.913778615" watchObservedRunningTime="2026-08-04 00:03:37.03522874 +0000 UTC m=+82.915676287" Aug 04 00:03:37.910901 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:37.910871 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85ffd74c56-qkg67" Aug 04 00:03:38.213727 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:38.213645 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:38.315837 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:38.315787 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:38.414501 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:38.414457 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:38.515726 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:38.515651 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:39.591498 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:39.591461 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:39.644581 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:39.644544 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:40.701448 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:40.701416 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:40.803237 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:40.803191 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:41.128552 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:41.128518 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:41.227075 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:41.227033 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:44.596145 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:44.596110 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:44.697948 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:44.697906 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:44.842229 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:44.842195 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:44.927910 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:44.927872 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:45.867989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:45.867950 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:45.968923 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:45.968875 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:47.003649 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:47.003623 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:47.004106 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:47.003692 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:03:49.874213 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:49.874178 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:03:49.975473 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:03:49.975438 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:03.996263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:03.996230 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dnb4v" Aug 04 00:04:04.876441 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:04.876400 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:04.944406 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:04.944372 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:07.008995 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:07.008965 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:04:07.012745 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:07.012721 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-db9668795-zn7bm" Aug 04 00:04:16.554448 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:16.554410 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:16.611096 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:16.611065 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:24.399112 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:24.399067 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:28.941569 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:28.941528 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:29.025195 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:29.025158 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:41.515641 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:41.515600 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:41.617366 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:41.617328 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:49.212883 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:49.212841 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:49.313955 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:49.313915 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:04:55.106332 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:04:55.106290 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:00.229902 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:00.229865 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:00.330213 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:00.330175 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:00.378487 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:00.378454 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:56.737120 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:56.737067 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:56.793523 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:56.793478 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:59.081545 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:59.081514 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:59.184130 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:59.184095 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:59.284969 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:59.284939 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:59.814669 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:59.814621 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:05:59.878978 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:05:59.878945 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:08.688435 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:08.688372 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:08.788473 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:08.788438 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:09.512178 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:09.512136 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:09.561256 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:09.561214 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:09.617922 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:09.617892 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:17.582753 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:17.582710 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:17.684225 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:17.684190 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:56.578021 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:56.577981 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:56.672001 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:56.671970 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:06:57.318396 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.318364 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw"] Aug 04 00:06:57.321566 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.321549 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.324036 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.324014 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 00:06:57.324134 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.324036 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 00:06:57.324990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.324973 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b7jgn\"" Aug 04 00:06:57.330457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.330437 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw"] Aug 04 00:06:57.333132 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.333111 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.333227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.333161 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95wqr\" (UniqueName: \"kubernetes.io/projected/945bc5d6-5039-4bc6-8564-bd74d6edc059-kube-api-access-95wqr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.333227 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.333185 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.433634 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.433607 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.433763 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.433648 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95wqr\" (UniqueName: \"kubernetes.io/projected/945bc5d6-5039-4bc6-8564-bd74d6edc059-kube-api-access-95wqr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.433763 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.433674 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.434027 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.434008 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.434063 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.434038 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.443187 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.443161 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95wqr\" (UniqueName: \"kubernetes.io/projected/945bc5d6-5039-4bc6-8564-bd74d6edc059-kube-api-access-95wqr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.630823 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.630778 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:06:57.754309 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:57.754279 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw"] Aug 04 00:06:57.757225 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:06:57.757200 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945bc5d6_5039_4bc6_8564_bd74d6edc059.slice/crio-5c6ddeb19e0d3a7907b4130801f597b5dda59f3444a9f4a6ca4d20b1bf8ff73e WatchSource:0}: Error finding container 5c6ddeb19e0d3a7907b4130801f597b5dda59f3444a9f4a6ca4d20b1bf8ff73e: Status 404 returned error can't find the container with id 5c6ddeb19e0d3a7907b4130801f597b5dda59f3444a9f4a6ca4d20b1bf8ff73e Aug 04 00:06:58.694538 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:06:58.694499 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" event={"ID":"945bc5d6-5039-4bc6-8564-bd74d6edc059","Type":"ContainerStarted","Data":"5c6ddeb19e0d3a7907b4130801f597b5dda59f3444a9f4a6ca4d20b1bf8ff73e"} Aug 04 00:07:04.717725 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:04.717678 2551 generic.go:356] "Generic (PLEG): container finished" podID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerID="b86a51a6e09f435bacd53b081bebbf79650b486ab36791ab5cb247921f3b18fc" exitCode=0 Aug 04 00:07:04.717725 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:04.717722 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" event={"ID":"945bc5d6-5039-4bc6-8564-bd74d6edc059","Type":"ContainerDied","Data":"b86a51a6e09f435bacd53b081bebbf79650b486ab36791ab5cb247921f3b18fc"} Aug 04 00:07:07.730432 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:07.730399 2551 generic.go:356] "Generic (PLEG): container finished" podID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerID="c4872116a530cb6b1bd65493e980e46aff34b1a757dcb4f742fa61c865240c59" exitCode=0 Aug 04 00:07:07.730840 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:07.730489 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" event={"ID":"945bc5d6-5039-4bc6-8564-bd74d6edc059","Type":"ContainerDied","Data":"c4872116a530cb6b1bd65493e980e46aff34b1a757dcb4f742fa61c865240c59"} Aug 04 00:07:14.551398 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:14.551376 2551 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 00:07:14.758317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:14.758239 2551 generic.go:356] "Generic (PLEG): container finished" podID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerID="61419c8f449c2f08bf38f47701484572a7e37cfff36ab8e22412be3c588df573" exitCode=0 Aug 04 00:07:14.758317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:14.758277 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" event={"ID":"945bc5d6-5039-4bc6-8564-bd74d6edc059","Type":"ContainerDied","Data":"61419c8f449c2f08bf38f47701484572a7e37cfff36ab8e22412be3c588df573"} Aug 04 00:07:15.882224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:15.882199 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:07:15.989809 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:15.989771 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-util\") pod \"945bc5d6-5039-4bc6-8564-bd74d6edc059\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " Aug 04 00:07:15.989986 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:15.989907 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-bundle\") pod \"945bc5d6-5039-4bc6-8564-bd74d6edc059\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " Aug 04 00:07:15.989986 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:15.989940 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95wqr\" (UniqueName: \"kubernetes.io/projected/945bc5d6-5039-4bc6-8564-bd74d6edc059-kube-api-access-95wqr\") pod \"945bc5d6-5039-4bc6-8564-bd74d6edc059\" (UID: \"945bc5d6-5039-4bc6-8564-bd74d6edc059\") " Aug 04 00:07:15.990501 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:15.990473 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-bundle" (OuterVolumeSpecName: "bundle") pod "945bc5d6-5039-4bc6-8564-bd74d6edc059" (UID: "945bc5d6-5039-4bc6-8564-bd74d6edc059"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:07:15.992073 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:15.992046 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945bc5d6-5039-4bc6-8564-bd74d6edc059-kube-api-access-95wqr" (OuterVolumeSpecName: "kube-api-access-95wqr") pod "945bc5d6-5039-4bc6-8564-bd74d6edc059" (UID: "945bc5d6-5039-4bc6-8564-bd74d6edc059"). InnerVolumeSpecName "kube-api-access-95wqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:07:15.994485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:15.994458 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-util" (OuterVolumeSpecName: "util") pod "945bc5d6-5039-4bc6-8564-bd74d6edc059" (UID: "945bc5d6-5039-4bc6-8564-bd74d6edc059"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:07:16.090789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:16.090693 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-util\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:16.090789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:16.090731 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/945bc5d6-5039-4bc6-8564-bd74d6edc059-bundle\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:16.090789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:16.090744 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95wqr\" (UniqueName: \"kubernetes.io/projected/945bc5d6-5039-4bc6-8564-bd74d6edc059-kube-api-access-95wqr\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:16.766655 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:16.766625 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" Aug 04 00:07:16.766830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:16.766625 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5grw" event={"ID":"945bc5d6-5039-4bc6-8564-bd74d6edc059","Type":"ContainerDied","Data":"5c6ddeb19e0d3a7907b4130801f597b5dda59f3444a9f4a6ca4d20b1bf8ff73e"} Aug 04 00:07:16.766830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:16.766741 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c6ddeb19e0d3a7907b4130801f597b5dda59f3444a9f4a6ca4d20b1bf8ff73e" Aug 04 00:07:18.301149 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:18.301108 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:19.837442 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:19.837402 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:20.215624 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215591 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j"] Aug 04 00:07:20.215891 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215879 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerName="extract" Aug 04 00:07:20.215938 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215893 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerName="extract" Aug 04 00:07:20.215938 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215910 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerName="util" Aug 04 00:07:20.215938 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215915 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerName="util" Aug 04 00:07:20.215938 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215923 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerName="pull" Aug 04 00:07:20.215938 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215929 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerName="pull" Aug 04 00:07:20.216086 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.215973 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="945bc5d6-5039-4bc6-8564-bd74d6edc059" containerName="extract" Aug 04 00:07:20.257701 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.257674 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j"] Aug 04 00:07:20.257848 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.257765 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.264111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.263702 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-wktm5\"" Aug 04 00:07:20.264111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.263850 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Aug 04 00:07:20.264244 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.264132 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Aug 04 00:07:20.326920 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.326890 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a03ed02c-7898-4417-abf0-c7046811d74d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pzb2j\" (UID: \"a03ed02c-7898-4417-abf0-c7046811d74d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.327088 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.326931 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxj4w\" (UniqueName: \"kubernetes.io/projected/a03ed02c-7898-4417-abf0-c7046811d74d-kube-api-access-dxj4w\") pod \"cert-manager-operator-controller-manager-55c444bf86-pzb2j\" (UID: \"a03ed02c-7898-4417-abf0-c7046811d74d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.428089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.428049 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a03ed02c-7898-4417-abf0-c7046811d74d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pzb2j\" (UID: \"a03ed02c-7898-4417-abf0-c7046811d74d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.428089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.428092 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxj4w\" (UniqueName: \"kubernetes.io/projected/a03ed02c-7898-4417-abf0-c7046811d74d-kube-api-access-dxj4w\") pod \"cert-manager-operator-controller-manager-55c444bf86-pzb2j\" (UID: \"a03ed02c-7898-4417-abf0-c7046811d74d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.428453 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.428432 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a03ed02c-7898-4417-abf0-c7046811d74d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pzb2j\" (UID: \"a03ed02c-7898-4417-abf0-c7046811d74d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.436280 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.436248 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxj4w\" (UniqueName: \"kubernetes.io/projected/a03ed02c-7898-4417-abf0-c7046811d74d-kube-api-access-dxj4w\") pod \"cert-manager-operator-controller-manager-55c444bf86-pzb2j\" (UID: \"a03ed02c-7898-4417-abf0-c7046811d74d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.567064 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.567000 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" Aug 04 00:07:20.726094 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.726042 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j"] Aug 04 00:07:20.728567 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:07:20.728539 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03ed02c_7898_4417_abf0_c7046811d74d.slice/crio-95e0c22ee6f282dc7b968a8335b5f9b70466be7f6c88d7c79c2f5b96dacf0f6f WatchSource:0}: Error finding container 95e0c22ee6f282dc7b968a8335b5f9b70466be7f6c88d7c79c2f5b96dacf0f6f: Status 404 returned error can't find the container with id 95e0c22ee6f282dc7b968a8335b5f9b70466be7f6c88d7c79c2f5b96dacf0f6f Aug 04 00:07:20.731347 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.731327 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 00:07:20.782591 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:20.782562 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" event={"ID":"a03ed02c-7898-4417-abf0-c7046811d74d","Type":"ContainerStarted","Data":"95e0c22ee6f282dc7b968a8335b5f9b70466be7f6c88d7c79c2f5b96dacf0f6f"} Aug 04 00:07:21.926421 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:21.926379 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:22.028900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:22.028864 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:26.806721 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:26.806632 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" event={"ID":"a03ed02c-7898-4417-abf0-c7046811d74d","Type":"ContainerStarted","Data":"6b5d6e9b6d4d3b42ef945e69530a63d408f69e34c6fafea52a3eb58ef5f9959a"} Aug 04 00:07:26.831887 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:26.831841 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pzb2j" podStartSLOduration=1.026950486 podStartE2EDuration="6.831825736s" totalImagesPullingTime="5.80487525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:07:26.829570012 +0000 UTC m=+312.710017553" watchObservedRunningTime="2026-08-04 00:07:26.831825736 +0000 UTC m=+312.712273275" Aug 04 00:07:26.918595 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:26.918562 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:27.030868 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:27.030828 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:27.156044 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:27.155995 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:27.246458 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:27.246415 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:27.487476 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:27.487398 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:27.586582 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:27.586539 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:28.537533 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.537497 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z"] Aug 04 00:07:28.540975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.540958 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.543656 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.543636 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b7jgn\"" Aug 04 00:07:28.543779 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.543673 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 00:07:28.544579 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.544564 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 00:07:28.549200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.549174 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z"] Aug 04 00:07:28.691188 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.691156 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwv79\" (UniqueName: \"kubernetes.io/projected/de960e9e-5c91-4fa5-83c7-f760278065ef-kube-api-access-pwv79\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.691335 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.691204 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.691335 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.691298 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.791785 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.791713 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.791785 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.791760 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.791965 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.791840 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwv79\" (UniqueName: \"kubernetes.io/projected/de960e9e-5c91-4fa5-83c7-f760278065ef-kube-api-access-pwv79\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.792168 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.792150 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.792208 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.792148 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.802142 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.802119 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwv79\" (UniqueName: \"kubernetes.io/projected/de960e9e-5c91-4fa5-83c7-f760278065ef-kube-api-access-pwv79\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.849998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.849967 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:28.973407 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:28.973278 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z"] Aug 04 00:07:28.975989 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:07:28.975961 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde960e9e_5c91_4fa5_83c7_f760278065ef.slice/crio-774d349c2822757f6188f1ba6b48240a690277b2f840e1a2c72d8881b9938e82 WatchSource:0}: Error finding container 774d349c2822757f6188f1ba6b48240a690277b2f840e1a2c72d8881b9938e82: Status 404 returned error can't find the container with id 774d349c2822757f6188f1ba6b48240a690277b2f840e1a2c72d8881b9938e82 Aug 04 00:07:29.818730 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:29.818692 2551 generic.go:356] "Generic (PLEG): container finished" podID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerID="ae4e2989c69bf8bb9f27d3c4d7c233817c58265403ee7aa3f5cf2d4a856d0fb4" exitCode=0 Aug 04 00:07:29.819067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:29.818776 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" event={"ID":"de960e9e-5c91-4fa5-83c7-f760278065ef","Type":"ContainerDied","Data":"ae4e2989c69bf8bb9f27d3c4d7c233817c58265403ee7aa3f5cf2d4a856d0fb4"} Aug 04 00:07:29.819067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:29.818840 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" event={"ID":"de960e9e-5c91-4fa5-83c7-f760278065ef","Type":"ContainerStarted","Data":"774d349c2822757f6188f1ba6b48240a690277b2f840e1a2c72d8881b9938e82"} Aug 04 00:07:29.929823 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:29.929778 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:30.063107 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.063029 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:30.442870 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.442835 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-chgh6"] Aug 04 00:07:30.446047 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.446028 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.448431 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.448396 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 04 00:07:30.449441 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.449425 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wg9qd\"" Aug 04 00:07:30.449545 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.449429 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 04 00:07:30.453474 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.453455 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-chgh6"] Aug 04 00:07:30.609225 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.609195 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hhn6\" (UniqueName: \"kubernetes.io/projected/ee75ca49-93b2-4239-9ff2-6d81e0cd4893-kube-api-access-9hhn6\") pod \"cert-manager-cainjector-559cdd8c87-chgh6\" (UID: \"ee75ca49-93b2-4239-9ff2-6d81e0cd4893\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.609400 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.609251 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee75ca49-93b2-4239-9ff2-6d81e0cd4893-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-chgh6\" (UID: \"ee75ca49-93b2-4239-9ff2-6d81e0cd4893\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.710617 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.710517 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hhn6\" (UniqueName: \"kubernetes.io/projected/ee75ca49-93b2-4239-9ff2-6d81e0cd4893-kube-api-access-9hhn6\") pod \"cert-manager-cainjector-559cdd8c87-chgh6\" (UID: \"ee75ca49-93b2-4239-9ff2-6d81e0cd4893\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.710617 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.710600 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee75ca49-93b2-4239-9ff2-6d81e0cd4893-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-chgh6\" (UID: \"ee75ca49-93b2-4239-9ff2-6d81e0cd4893\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.719674 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.719647 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee75ca49-93b2-4239-9ff2-6d81e0cd4893-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-chgh6\" (UID: \"ee75ca49-93b2-4239-9ff2-6d81e0cd4893\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.719891 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.719872 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hhn6\" (UniqueName: \"kubernetes.io/projected/ee75ca49-93b2-4239-9ff2-6d81e0cd4893-kube-api-access-9hhn6\") pod \"cert-manager-cainjector-559cdd8c87-chgh6\" (UID: \"ee75ca49-93b2-4239-9ff2-6d81e0cd4893\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.765362 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.765331 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" Aug 04 00:07:30.897626 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:30.897599 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-chgh6"] Aug 04 00:07:30.899773 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:07:30.899733 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee75ca49_93b2_4239_9ff2_6d81e0cd4893.slice/crio-bfac4f6e76616e4c20b6ee98dbb5d577492267ce5c450f0b101e6ba3f2bd8a73 WatchSource:0}: Error finding container bfac4f6e76616e4c20b6ee98dbb5d577492267ce5c450f0b101e6ba3f2bd8a73: Status 404 returned error can't find the container with id bfac4f6e76616e4c20b6ee98dbb5d577492267ce5c450f0b101e6ba3f2bd8a73 Aug 04 00:07:31.829233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:31.829174 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" event={"ID":"ee75ca49-93b2-4239-9ff2-6d81e0cd4893","Type":"ContainerStarted","Data":"bfac4f6e76616e4c20b6ee98dbb5d577492267ce5c450f0b101e6ba3f2bd8a73"} Aug 04 00:07:33.838394 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:33.838362 2551 generic.go:356] "Generic (PLEG): container finished" podID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerID="c22e860c1e3615e4cf085d21488fc5558ce222bbbc43c2422a554ce01c7d93e0" exitCode=0 Aug 04 00:07:33.838766 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:33.838445 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" event={"ID":"de960e9e-5c91-4fa5-83c7-f760278065ef","Type":"ContainerDied","Data":"c22e860c1e3615e4cf085d21488fc5558ce222bbbc43c2422a554ce01c7d93e0"} Aug 04 00:07:34.844587 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:34.844553 2551 generic.go:356] "Generic (PLEG): container finished" podID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerID="77425f6aa966374171658604b915b606ad135e7eabe22325a4637b41df1b76c2" exitCode=0 Aug 04 00:07:34.844996 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:34.844645 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" event={"ID":"de960e9e-5c91-4fa5-83c7-f760278065ef","Type":"ContainerDied","Data":"77425f6aa966374171658604b915b606ad135e7eabe22325a4637b41df1b76c2"} Aug 04 00:07:35.968897 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:35.968871 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:36.062342 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.062310 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwv79\" (UniqueName: \"kubernetes.io/projected/de960e9e-5c91-4fa5-83c7-f760278065ef-kube-api-access-pwv79\") pod \"de960e9e-5c91-4fa5-83c7-f760278065ef\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " Aug 04 00:07:36.062516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.062371 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-util\") pod \"de960e9e-5c91-4fa5-83c7-f760278065ef\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " Aug 04 00:07:36.062516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.062412 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-bundle\") pod \"de960e9e-5c91-4fa5-83c7-f760278065ef\" (UID: \"de960e9e-5c91-4fa5-83c7-f760278065ef\") " Aug 04 00:07:36.062876 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.062843 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-bundle" (OuterVolumeSpecName: "bundle") pod "de960e9e-5c91-4fa5-83c7-f760278065ef" (UID: "de960e9e-5c91-4fa5-83c7-f760278065ef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:07:36.064422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.064396 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de960e9e-5c91-4fa5-83c7-f760278065ef-kube-api-access-pwv79" (OuterVolumeSpecName: "kube-api-access-pwv79") pod "de960e9e-5c91-4fa5-83c7-f760278065ef" (UID: "de960e9e-5c91-4fa5-83c7-f760278065ef"). InnerVolumeSpecName "kube-api-access-pwv79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:07:36.067112 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.067090 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-util" (OuterVolumeSpecName: "util") pod "de960e9e-5c91-4fa5-83c7-f760278065ef" (UID: "de960e9e-5c91-4fa5-83c7-f760278065ef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:07:36.163328 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.163304 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-bundle\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:36.163328 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.163328 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwv79\" (UniqueName: \"kubernetes.io/projected/de960e9e-5c91-4fa5-83c7-f760278065ef-kube-api-access-pwv79\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:36.163485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.163338 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de960e9e-5c91-4fa5-83c7-f760278065ef-util\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:36.854971 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.854930 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" event={"ID":"de960e9e-5c91-4fa5-83c7-f760278065ef","Type":"ContainerDied","Data":"774d349c2822757f6188f1ba6b48240a690277b2f840e1a2c72d8881b9938e82"} Aug 04 00:07:36.855112 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.854984 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="774d349c2822757f6188f1ba6b48240a690277b2f840e1a2c72d8881b9938e82" Aug 04 00:07:36.855112 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:36.854948 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4477z" Aug 04 00:07:37.671177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.671147 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:37.822975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.822939 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d2mpz"] Aug 04 00:07:37.823208 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.823196 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerName="extract" Aug 04 00:07:37.823255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.823209 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerName="extract" Aug 04 00:07:37.823255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.823219 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerName="util" Aug 04 00:07:37.823255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.823224 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerName="util" Aug 04 00:07:37.823255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.823230 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerName="pull" Aug 04 00:07:37.823255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.823235 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerName="pull" Aug 04 00:07:37.823417 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.823277 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="de960e9e-5c91-4fa5-83c7-f760278065ef" containerName="extract" Aug 04 00:07:37.826055 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.826039 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:37.828436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.828415 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-68x8b\"" Aug 04 00:07:37.835425 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.835407 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d2mpz"] Aug 04 00:07:37.978085 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.978017 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pn4d\" (UniqueName: \"kubernetes.io/projected/aa5667f3-6a90-4e3e-be10-3b6c7380cb36-kube-api-access-2pn4d\") pod \"cert-manager-55b4d954d5-d2mpz\" (UID: \"aa5667f3-6a90-4e3e-be10-3b6c7380cb36\") " pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:37.978085 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:37.978047 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa5667f3-6a90-4e3e-be10-3b6c7380cb36-bound-sa-token\") pod \"cert-manager-55b4d954d5-d2mpz\" (UID: \"aa5667f3-6a90-4e3e-be10-3b6c7380cb36\") " pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:38.079483 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:38.079452 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pn4d\" (UniqueName: \"kubernetes.io/projected/aa5667f3-6a90-4e3e-be10-3b6c7380cb36-kube-api-access-2pn4d\") pod \"cert-manager-55b4d954d5-d2mpz\" (UID: \"aa5667f3-6a90-4e3e-be10-3b6c7380cb36\") " pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:38.079483 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:38.079484 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa5667f3-6a90-4e3e-be10-3b6c7380cb36-bound-sa-token\") pod \"cert-manager-55b4d954d5-d2mpz\" (UID: \"aa5667f3-6a90-4e3e-be10-3b6c7380cb36\") " pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:38.088323 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:38.088292 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa5667f3-6a90-4e3e-be10-3b6c7380cb36-bound-sa-token\") pod \"cert-manager-55b4d954d5-d2mpz\" (UID: \"aa5667f3-6a90-4e3e-be10-3b6c7380cb36\") " pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:38.088479 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:38.088465 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pn4d\" (UniqueName: \"kubernetes.io/projected/aa5667f3-6a90-4e3e-be10-3b6c7380cb36-kube-api-access-2pn4d\") pod \"cert-manager-55b4d954d5-d2mpz\" (UID: \"aa5667f3-6a90-4e3e-be10-3b6c7380cb36\") " pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:38.134875 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:38.134846 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d2mpz" Aug 04 00:07:38.256652 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:38.256628 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d2mpz"] Aug 04 00:07:38.258658 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:07:38.258628 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa5667f3_6a90_4e3e_be10_3b6c7380cb36.slice/crio-88f01a5f29e8b96bd79bed5ceaea7d23ea2a5284f5eb0ade716bbc6087d5f121 WatchSource:0}: Error finding container 88f01a5f29e8b96bd79bed5ceaea7d23ea2a5284f5eb0ade716bbc6087d5f121: Status 404 returned error can't find the container with id 88f01a5f29e8b96bd79bed5ceaea7d23ea2a5284f5eb0ade716bbc6087d5f121 Aug 04 00:07:38.866097 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:38.866016 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d2mpz" event={"ID":"aa5667f3-6a90-4e3e-be10-3b6c7380cb36","Type":"ContainerStarted","Data":"88f01a5f29e8b96bd79bed5ceaea7d23ea2a5284f5eb0ade716bbc6087d5f121"} Aug 04 00:07:40.877073 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:40.877034 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d2mpz" event={"ID":"aa5667f3-6a90-4e3e-be10-3b6c7380cb36","Type":"ContainerStarted","Data":"aa9d86ce76049fa5eef8b10cabc7fb339a2c9c326fe8fe1988f28437b2ae9db7"} Aug 04 00:07:40.878575 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:40.878538 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" event={"ID":"ee75ca49-93b2-4239-9ff2-6d81e0cd4893","Type":"ContainerStarted","Data":"5e65c22d06c1467a9f105c1c719b07c10c9c25ff266dcdd58898d84ffeeda2b3"} Aug 04 00:07:40.902159 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:40.902111 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-d2mpz" podStartSLOduration=1.5454037980000002 podStartE2EDuration="3.902096574s" totalImagesPullingTime="2.356692776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:07:40.901556556 +0000 UTC m=+326.782004098" watchObservedRunningTime="2026-08-04 00:07:40.902096574 +0000 UTC m=+326.782544114" Aug 04 00:07:40.919942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:40.919899 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-chgh6" podStartSLOduration=1.211144526 podStartE2EDuration="10.919884846s" totalImagesPullingTime="9.70874032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:07:40.91855824 +0000 UTC m=+326.799005783" watchObservedRunningTime="2026-08-04 00:07:40.919884846 +0000 UTC m=+326.800332386" Aug 04 00:07:40.922131 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:40.922106 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:40.992085 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:40.992053 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:41.389554 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:41.389520 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:41.449258 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:41.449232 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:42.443156 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.443122 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99"] Aug 04 00:07:42.446444 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.446422 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.448926 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.448900 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 00:07:42.450013 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.449996 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lk7jr\"" Aug 04 00:07:42.450111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.450033 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 00:07:42.454979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.454959 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99"] Aug 04 00:07:42.615391 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.615351 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjplb\" (UniqueName: \"kubernetes.io/projected/d5399b99-3c84-457f-8018-837450a2920c-kube-api-access-vjplb\") pod \"openshift-lws-operator-fd8ccff4c-bcv99\" (UID: \"d5399b99-3c84-457f-8018-837450a2920c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.615391 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.615392 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5399b99-3c84-457f-8018-837450a2920c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bcv99\" (UID: \"d5399b99-3c84-457f-8018-837450a2920c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.716564 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.716483 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjplb\" (UniqueName: \"kubernetes.io/projected/d5399b99-3c84-457f-8018-837450a2920c-kube-api-access-vjplb\") pod \"openshift-lws-operator-fd8ccff4c-bcv99\" (UID: \"d5399b99-3c84-457f-8018-837450a2920c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.716564 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.716524 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5399b99-3c84-457f-8018-837450a2920c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bcv99\" (UID: \"d5399b99-3c84-457f-8018-837450a2920c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.716954 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.716932 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5399b99-3c84-457f-8018-837450a2920c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bcv99\" (UID: \"d5399b99-3c84-457f-8018-837450a2920c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.725950 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.725924 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjplb\" (UniqueName: \"kubernetes.io/projected/d5399b99-3c84-457f-8018-837450a2920c-kube-api-access-vjplb\") pod \"openshift-lws-operator-fd8ccff4c-bcv99\" (UID: \"d5399b99-3c84-457f-8018-837450a2920c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.756746 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.756723 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" Aug 04 00:07:42.879974 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.879899 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99"] Aug 04 00:07:42.882065 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:07:42.882036 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5399b99_3c84_457f_8018_837450a2920c.slice/crio-f1d6ec6928fff1eff4c53249fdba99627438d28457186d85577ab722df9a343e WatchSource:0}: Error finding container f1d6ec6928fff1eff4c53249fdba99627438d28457186d85577ab722df9a343e: Status 404 returned error can't find the container with id f1d6ec6928fff1eff4c53249fdba99627438d28457186d85577ab722df9a343e Aug 04 00:07:42.888647 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:42.888621 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" event={"ID":"d5399b99-3c84-457f-8018-837450a2920c","Type":"ContainerStarted","Data":"f1d6ec6928fff1eff4c53249fdba99627438d28457186d85577ab722df9a343e"} Aug 04 00:07:46.906691 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:46.906638 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" event={"ID":"d5399b99-3c84-457f-8018-837450a2920c","Type":"ContainerStarted","Data":"7af5aebe97b6a4cc733c4f800794e38acbb0b413cb05e72b5e835f47e12193b8"} Aug 04 00:07:46.923776 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:46.923728 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bcv99" podStartSLOduration=1.2861627869999999 podStartE2EDuration="4.923714499s" totalImagesPullingTime="3.637551712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:07:46.922254813 +0000 UTC m=+332.802702353" watchObservedRunningTime="2026-08-04 00:07:46.923714499 +0000 UTC m=+332.804162038" Aug 04 00:07:47.614906 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:47.614867 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:47.715651 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:47.715619 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:49.020289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.020254 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg"] Aug 04 00:07:49.023699 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.023682 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.026120 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.026097 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 00:07:49.026204 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.026143 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 00:07:49.027049 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.027034 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b7jgn\"" Aug 04 00:07:49.030783 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.030610 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg"] Aug 04 00:07:49.054245 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.054216 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.054379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.054264 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.054379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.054363 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhch\" (UniqueName: \"kubernetes.io/projected/93a04a24-7936-476b-af76-8f35479bc9a8-kube-api-access-9nhch\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.154927 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.154890 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.155097 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.154947 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhch\" (UniqueName: \"kubernetes.io/projected/93a04a24-7936-476b-af76-8f35479bc9a8-kube-api-access-9nhch\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.155097 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.154978 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.155291 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.155270 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.155356 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.155280 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.163771 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.163748 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhch\" (UniqueName: \"kubernetes.io/projected/93a04a24-7936-476b-af76-8f35479bc9a8-kube-api-access-9nhch\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.334010 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.333919 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:49.458094 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.458066 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg"] Aug 04 00:07:49.460220 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:07:49.460195 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a04a24_7936_476b_af76_8f35479bc9a8.slice/crio-dc0db009aa631a91634f6fdaf3d240b73f09213ddd30319fe57e689e93651527 WatchSource:0}: Error finding container dc0db009aa631a91634f6fdaf3d240b73f09213ddd30319fe57e689e93651527: Status 404 returned error can't find the container with id dc0db009aa631a91634f6fdaf3d240b73f09213ddd30319fe57e689e93651527 Aug 04 00:07:49.921063 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.921027 2551 generic.go:356] "Generic (PLEG): container finished" podID="93a04a24-7936-476b-af76-8f35479bc9a8" containerID="e8b93f7cca1b86122228caaea38a7212f6aca00ae5f695c28e37f102d0f2755e" exitCode=0 Aug 04 00:07:49.921245 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.921118 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" event={"ID":"93a04a24-7936-476b-af76-8f35479bc9a8","Type":"ContainerDied","Data":"e8b93f7cca1b86122228caaea38a7212f6aca00ae5f695c28e37f102d0f2755e"} Aug 04 00:07:49.921245 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.921198 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" event={"ID":"93a04a24-7936-476b-af76-8f35479bc9a8","Type":"ContainerStarted","Data":"dc0db009aa631a91634f6fdaf3d240b73f09213ddd30319fe57e689e93651527"} Aug 04 00:07:49.921769 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:49.921738 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:07:51.931120 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:51.931089 2551 generic.go:356] "Generic (PLEG): container finished" podID="93a04a24-7936-476b-af76-8f35479bc9a8" containerID="2fb1e052aa3eaf4188a34130f300eff93d754e01ead12625cbc903bbcdf130e5" exitCode=0 Aug 04 00:07:51.931459 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:51.931175 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" event={"ID":"93a04a24-7936-476b-af76-8f35479bc9a8","Type":"ContainerDied","Data":"2fb1e052aa3eaf4188a34130f300eff93d754e01ead12625cbc903bbcdf130e5"} Aug 04 00:07:52.936752 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:52.936720 2551 generic.go:356] "Generic (PLEG): container finished" podID="93a04a24-7936-476b-af76-8f35479bc9a8" containerID="8035873062f282c34c47f54f9679b940dd32808ca3869c7f0e9ca7216a6a0a42" exitCode=0 Aug 04 00:07:52.937129 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:52.936821 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" event={"ID":"93a04a24-7936-476b-af76-8f35479bc9a8","Type":"ContainerDied","Data":"8035873062f282c34c47f54f9679b940dd32808ca3869c7f0e9ca7216a6a0a42"} Aug 04 00:07:54.068907 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.068881 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:54.090168 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.090146 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-util\") pod \"93a04a24-7936-476b-af76-8f35479bc9a8\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " Aug 04 00:07:54.090270 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.090245 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nhch\" (UniqueName: \"kubernetes.io/projected/93a04a24-7936-476b-af76-8f35479bc9a8-kube-api-access-9nhch\") pod \"93a04a24-7936-476b-af76-8f35479bc9a8\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " Aug 04 00:07:54.090318 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.090285 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-bundle\") pod \"93a04a24-7936-476b-af76-8f35479bc9a8\" (UID: \"93a04a24-7936-476b-af76-8f35479bc9a8\") " Aug 04 00:07:54.091070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.091041 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-bundle" (OuterVolumeSpecName: "bundle") pod "93a04a24-7936-476b-af76-8f35479bc9a8" (UID: "93a04a24-7936-476b-af76-8f35479bc9a8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:07:54.092471 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.092442 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93a04a24-7936-476b-af76-8f35479bc9a8-kube-api-access-9nhch" (OuterVolumeSpecName: "kube-api-access-9nhch") pod "93a04a24-7936-476b-af76-8f35479bc9a8" (UID: "93a04a24-7936-476b-af76-8f35479bc9a8"). InnerVolumeSpecName "kube-api-access-9nhch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:07:54.099079 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.099035 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-util" (OuterVolumeSpecName: "util") pod "93a04a24-7936-476b-af76-8f35479bc9a8" (UID: "93a04a24-7936-476b-af76-8f35479bc9a8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:07:54.191496 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.191467 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nhch\" (UniqueName: \"kubernetes.io/projected/93a04a24-7936-476b-af76-8f35479bc9a8-kube-api-access-9nhch\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:54.191496 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.191493 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-bundle\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:54.191657 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.191502 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/93a04a24-7936-476b-af76-8f35479bc9a8-util\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:07:54.947108 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.947027 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" event={"ID":"93a04a24-7936-476b-af76-8f35479bc9a8","Type":"ContainerDied","Data":"dc0db009aa631a91634f6fdaf3d240b73f09213ddd30319fe57e689e93651527"} Aug 04 00:07:54.947108 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.947047 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52rdg" Aug 04 00:07:54.947108 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:07:54.947074 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc0db009aa631a91634f6fdaf3d240b73f09213ddd30319fe57e689e93651527" Aug 04 00:08:03.553989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:03.553944 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:05.158257 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:05.158207 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:05.253874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:05.253830 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:05.814013 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:05.813974 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.011648 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.011604 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.116129 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.116036 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.144974 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.144946 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c"] Aug 04 00:08:06.145248 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.145235 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93a04a24-7936-476b-af76-8f35479bc9a8" containerName="util" Aug 04 00:08:06.145289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.145249 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="93a04a24-7936-476b-af76-8f35479bc9a8" containerName="util" Aug 04 00:08:06.145289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.145257 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93a04a24-7936-476b-af76-8f35479bc9a8" containerName="pull" Aug 04 00:08:06.145289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.145263 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="93a04a24-7936-476b-af76-8f35479bc9a8" containerName="pull" Aug 04 00:08:06.145289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.145270 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93a04a24-7936-476b-af76-8f35479bc9a8" containerName="extract" Aug 04 00:08:06.145289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.145276 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="93a04a24-7936-476b-af76-8f35479bc9a8" containerName="extract" Aug 04 00:08:06.145482 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.145327 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="93a04a24-7936-476b-af76-8f35479bc9a8" containerName="extract" Aug 04 00:08:06.151510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.151486 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.154160 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.154136 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 00:08:06.154263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.154136 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rp998\"" Aug 04 00:08:06.154263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.154212 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 00:08:06.154263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.154213 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 00:08:06.154263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.154239 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 00:08:06.169674 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.169653 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c"] Aug 04 00:08:06.180325 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.180305 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0708418b-6a3e-44f6-b49a-4408dc108661-tmp\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.180444 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.180344 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0708418b-6a3e-44f6-b49a-4408dc108661-manifests\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.180444 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.180388 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0708418b-6a3e-44f6-b49a-4408dc108661-webhook-cert\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.180558 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.180452 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0708418b-6a3e-44f6-b49a-4408dc108661-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.180558 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.180496 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvg69\" (UniqueName: \"kubernetes.io/projected/0708418b-6a3e-44f6-b49a-4408dc108661-kube-api-access-zvg69\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.281318 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.281281 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0708418b-6a3e-44f6-b49a-4408dc108661-webhook-cert\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.281512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.281336 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0708418b-6a3e-44f6-b49a-4408dc108661-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.281512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.281364 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvg69\" (UniqueName: \"kubernetes.io/projected/0708418b-6a3e-44f6-b49a-4408dc108661-kube-api-access-zvg69\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.281512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.281397 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0708418b-6a3e-44f6-b49a-4408dc108661-tmp\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.281512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.281419 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0708418b-6a3e-44f6-b49a-4408dc108661-manifests\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.281843 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.281783 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0708418b-6a3e-44f6-b49a-4408dc108661-manifests\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.283769 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.283741 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0708418b-6a3e-44f6-b49a-4408dc108661-tmp\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.283993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.283963 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0708418b-6a3e-44f6-b49a-4408dc108661-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.284036 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.283963 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0708418b-6a3e-44f6-b49a-4408dc108661-webhook-cert\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.289958 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.289934 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.292166 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.292137 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvg69\" (UniqueName: \"kubernetes.io/projected/0708418b-6a3e-44f6-b49a-4408dc108661-kube-api-access-zvg69\") pod \"opendatahub-operator-controller-manager-8594b76987-wqh9c\" (UID: \"0708418b-6a3e-44f6-b49a-4408dc108661\") " pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.405943 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.405905 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.461432 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.461399 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:06.496324 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.496291 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.592996 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.592972 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c"] Aug 04 00:08:06.595135 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:08:06.595110 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0708418b_6a3e_44f6_b49a_4408dc108661.slice/crio-a9fdda4cedd35e568a1dcc4f7bd6d75bd861b4fba6e3c88897dbcb01cc39915b WatchSource:0}: Error finding container a9fdda4cedd35e568a1dcc4f7bd6d75bd861b4fba6e3c88897dbcb01cc39915b: Status 404 returned error can't find the container with id a9fdda4cedd35e568a1dcc4f7bd6d75bd861b4fba6e3c88897dbcb01cc39915b Aug 04 00:08:06.598066 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.598039 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.696537 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.696432 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.797166 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.797131 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.897044 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.896964 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:06.996508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:06.996434 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" event={"ID":"0708418b-6a3e-44f6-b49a-4408dc108661","Type":"ContainerStarted","Data":"a9fdda4cedd35e568a1dcc4f7bd6d75bd861b4fba6e3c88897dbcb01cc39915b"} Aug 04 00:08:10.011459 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:10.011421 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" event={"ID":"0708418b-6a3e-44f6-b49a-4408dc108661","Type":"ContainerStarted","Data":"07c70c5375864e14ed599201f6411561a31f462ee153515bf822e2648f112dcc"} Aug 04 00:08:11.017263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:11.017183 2551 generic.go:356] "Generic (PLEG): container finished" podID="0708418b-6a3e-44f6-b49a-4408dc108661" containerID="07c70c5375864e14ed599201f6411561a31f462ee153515bf822e2648f112dcc" exitCode=0 Aug 04 00:08:11.017263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:11.017250 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" event={"ID":"0708418b-6a3e-44f6-b49a-4408dc108661","Type":"ContainerDied","Data":"07c70c5375864e14ed599201f6411561a31f462ee153515bf822e2648f112dcc"} Aug 04 00:08:12.024291 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:12.024258 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" event={"ID":"0708418b-6a3e-44f6-b49a-4408dc108661","Type":"ContainerStarted","Data":"931db8b952c4c2adc6974e179268dbc2393ea7aace51e30e9390d45e8049604b"} Aug 04 00:08:12.024708 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:12.024400 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:12.048609 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:12.048556 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" podStartSLOduration=1.615182026 podStartE2EDuration="6.048531852s" totalImagesPullingTime="3.367921382s" totalInitContainerRuntime="1.065428444s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:08:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:08:12.047409287 +0000 UTC m=+357.927856840" watchObservedRunningTime="2026-08-04 00:08:12.048531852 +0000 UTC m=+357.928979404" Aug 04 00:08:14.125515 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:14.125455 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:14.244380 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:14.244348 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:22.395535 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:22.395485 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:22.452063 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:22.452028 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:22.540737 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:22.540703 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:23.030508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:23.030479 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8594b76987-wqh9c" Aug 04 00:08:23.148716 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:23.148677 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:23.240878 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:23.240843 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:24.645132 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:24.645088 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:24.747309 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:24.747270 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:25.283874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.283832 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:25.320015 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.319981 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp"] Aug 04 00:08:25.330634 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.330610 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.335611 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.335589 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Aug 04 00:08:25.336013 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.335995 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 00:08:25.336516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.336501 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Aug 04 00:08:25.336666 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.336647 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 00:08:25.340052 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.340032 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp"] Aug 04 00:08:25.408033 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.407999 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2fvbv"] Aug 04 00:08:25.411443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.411423 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.413994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.413972 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Aug 04 00:08:25.414150 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.414120 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-l6nx4\"" Aug 04 00:08:25.415526 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.415479 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:25.422880 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.422856 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2fvbv"] Aug 04 00:08:25.438637 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.438613 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e129f437-cc33-4360-b132-927ae8b12083-tls-certs\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.438756 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.438694 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e129f437-cc33-4360-b132-927ae8b12083-tmp\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.438830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.438783 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6fxq\" (UniqueName: \"kubernetes.io/projected/e129f437-cc33-4360-b132-927ae8b12083-kube-api-access-v6fxq\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.490210 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.490172 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:25.539426 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.539353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf451cea-1e4b-4100-9ebd-8dc323e00e92-cert\") pod \"odh-model-controller-7998c67dc4-2fvbv\" (UID: \"bf451cea-1e4b-4100-9ebd-8dc323e00e92\") " pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.539426 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.539386 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6zgr\" (UniqueName: \"kubernetes.io/projected/bf451cea-1e4b-4100-9ebd-8dc323e00e92-kube-api-access-t6zgr\") pod \"odh-model-controller-7998c67dc4-2fvbv\" (UID: \"bf451cea-1e4b-4100-9ebd-8dc323e00e92\") " pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.539426 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.539415 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e129f437-cc33-4360-b132-927ae8b12083-tls-certs\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.539610 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.539474 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e129f437-cc33-4360-b132-927ae8b12083-tmp\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.539610 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.539502 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6fxq\" (UniqueName: \"kubernetes.io/projected/e129f437-cc33-4360-b132-927ae8b12083-kube-api-access-v6fxq\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.541625 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.541595 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e129f437-cc33-4360-b132-927ae8b12083-tmp\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.541930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.541910 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e129f437-cc33-4360-b132-927ae8b12083-tls-certs\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.548358 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.548334 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6fxq\" (UniqueName: \"kubernetes.io/projected/e129f437-cc33-4360-b132-927ae8b12083-kube-api-access-v6fxq\") pod \"kube-auth-proxy-57f7cf7b5f-xspfp\" (UID: \"e129f437-cc33-4360-b132-927ae8b12083\") " pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.586112 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.586076 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:25.640241 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.640216 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf451cea-1e4b-4100-9ebd-8dc323e00e92-cert\") pod \"odh-model-controller-7998c67dc4-2fvbv\" (UID: \"bf451cea-1e4b-4100-9ebd-8dc323e00e92\") " pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.640380 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.640252 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6zgr\" (UniqueName: \"kubernetes.io/projected/bf451cea-1e4b-4100-9ebd-8dc323e00e92-kube-api-access-t6zgr\") pod \"odh-model-controller-7998c67dc4-2fvbv\" (UID: \"bf451cea-1e4b-4100-9ebd-8dc323e00e92\") " pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.640380 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.640358 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" Aug 04 00:08:25.642530 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.642506 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf451cea-1e4b-4100-9ebd-8dc323e00e92-cert\") pod \"odh-model-controller-7998c67dc4-2fvbv\" (UID: \"bf451cea-1e4b-4100-9ebd-8dc323e00e92\") " pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.647641 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.647621 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6zgr\" (UniqueName: \"kubernetes.io/projected/bf451cea-1e4b-4100-9ebd-8dc323e00e92-kube-api-access-t6zgr\") pod \"odh-model-controller-7998c67dc4-2fvbv\" (UID: \"bf451cea-1e4b-4100-9ebd-8dc323e00e92\") " pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.687044 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.687001 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:25.723230 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.723196 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:25.769710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.769683 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp"] Aug 04 00:08:25.771842 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:08:25.771784 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode129f437_cc33_4360_b132_927ae8b12083.slice/crio-e2eda28b293d47ef322cd2fa2acaadf6dd48766e77bf524f44ede1608a0e8635 WatchSource:0}: Error finding container e2eda28b293d47ef322cd2fa2acaadf6dd48766e77bf524f44ede1608a0e8635: Status 404 returned error can't find the container with id e2eda28b293d47ef322cd2fa2acaadf6dd48766e77bf524f44ede1608a0e8635 Aug 04 00:08:25.772240 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.772212 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:25.856548 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:25.856519 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2fvbv"] Aug 04 00:08:25.860186 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:08:25.860157 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf451cea_1e4b_4100_9ebd_8dc323e00e92.slice/crio-eddd016e4cef008141a6edab17a18d588eaed532ca427318de230753e527ded2 WatchSource:0}: Error finding container eddd016e4cef008141a6edab17a18d588eaed532ca427318de230753e527ded2: Status 404 returned error can't find the container with id eddd016e4cef008141a6edab17a18d588eaed532ca427318de230753e527ded2 Aug 04 00:08:26.087307 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.087223 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" event={"ID":"e129f437-cc33-4360-b132-927ae8b12083","Type":"ContainerStarted","Data":"e2eda28b293d47ef322cd2fa2acaadf6dd48766e77bf524f44ede1608a0e8635"} Aug 04 00:08:26.088552 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.088516 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" event={"ID":"bf451cea-1e4b-4100-9ebd-8dc323e00e92","Type":"ContainerStarted","Data":"eddd016e4cef008141a6edab17a18d588eaed532ca427318de230753e527ded2"} Aug 04 00:08:26.129686 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.129663 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:26.141993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.141974 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-npdcp"] Aug 04 00:08:26.146602 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.146580 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.150508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.150492 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qj4kg\"" Aug 04 00:08:26.154764 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.154745 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Aug 04 00:08:26.164880 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.164860 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-npdcp"] Aug 04 00:08:26.225002 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.224958 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:26.245947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.245916 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2326363c-2143-4dec-8592-e5fdd7ab6dac-tls-certs\") pod \"model-serving-api-54d89bd5fc-npdcp\" (UID: \"2326363c-2143-4dec-8592-e5fdd7ab6dac\") " pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.246058 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.245978 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggfnt\" (UniqueName: \"kubernetes.io/projected/2326363c-2143-4dec-8592-e5fdd7ab6dac-kube-api-access-ggfnt\") pod \"model-serving-api-54d89bd5fc-npdcp\" (UID: \"2326363c-2143-4dec-8592-e5fdd7ab6dac\") " pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.346881 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.346779 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2326363c-2143-4dec-8592-e5fdd7ab6dac-tls-certs\") pod \"model-serving-api-54d89bd5fc-npdcp\" (UID: \"2326363c-2143-4dec-8592-e5fdd7ab6dac\") " pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.346881 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.346867 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggfnt\" (UniqueName: \"kubernetes.io/projected/2326363c-2143-4dec-8592-e5fdd7ab6dac-kube-api-access-ggfnt\") pod \"model-serving-api-54d89bd5fc-npdcp\" (UID: \"2326363c-2143-4dec-8592-e5fdd7ab6dac\") " pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.349203 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.349182 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2326363c-2143-4dec-8592-e5fdd7ab6dac-tls-certs\") pod \"model-serving-api-54d89bd5fc-npdcp\" (UID: \"2326363c-2143-4dec-8592-e5fdd7ab6dac\") " pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.360738 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.360712 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggfnt\" (UniqueName: \"kubernetes.io/projected/2326363c-2143-4dec-8592-e5fdd7ab6dac-kube-api-access-ggfnt\") pod \"model-serving-api-54d89bd5fc-npdcp\" (UID: \"2326363c-2143-4dec-8592-e5fdd7ab6dac\") " pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.457525 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.457491 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:26.567352 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.567273 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:26.604377 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.604305 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z"] Aug 04 00:08:26.608033 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.608009 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.610595 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.610518 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 00:08:26.610835 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.610523 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 00:08:26.610835 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.610819 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v8qwf\"" Aug 04 00:08:26.610980 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.610537 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 00:08:26.610980 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.610640 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 00:08:26.621093 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.621032 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-npdcp"] Aug 04 00:08:26.621765 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.621584 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z"] Aug 04 00:08:26.627608 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:08:26.627584 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2326363c_2143_4dec_8592_e5fdd7ab6dac.slice/crio-bd347d720203a95b935c576aa6a4bde4af0f93c41f5133c929f752ad2d55883f WatchSource:0}: Error finding container bd347d720203a95b935c576aa6a4bde4af0f93c41f5133c929f752ad2d55883f: Status 404 returned error can't find the container with id bd347d720203a95b935c576aa6a4bde4af0f93c41f5133c929f752ad2d55883f Aug 04 00:08:26.645111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.645085 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:26.712168 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.712137 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:26.750072 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.750039 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.750218 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.750086 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.750218 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.750117 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.750218 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.750171 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.750218 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.750211 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.750376 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.750240 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4k8v\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-kube-api-access-k4k8v\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.750376 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.750286 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.850911 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.850878 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.850911 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.850914 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.851155 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.850945 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.851155 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.850977 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.851155 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.851001 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.851155 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.851031 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4k8v\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-kube-api-access-k4k8v\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.851155 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.851059 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.851740 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.851691 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.853912 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.853878 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.854024 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.853956 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.854024 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.853982 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.854148 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.854125 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.859757 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.859699 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4k8v\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-kube-api-access-k4k8v\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.859875 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.859770 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7j49z\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:26.921871 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:26.921843 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:27.106018 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:27.105978 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" event={"ID":"2326363c-2143-4dec-8592-e5fdd7ab6dac","Type":"ContainerStarted","Data":"bd347d720203a95b935c576aa6a4bde4af0f93c41f5133c929f752ad2d55883f"} Aug 04 00:08:27.128660 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:27.128232 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z"] Aug 04 00:08:27.132089 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:08:27.131144 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4054c4_d6bf_4c47_96b0_aa6c818722f0.slice/crio-14caf970d24ccff0e5fa020045d5da33f3778e23111639e870fbca962adcdb3e WatchSource:0}: Error finding container 14caf970d24ccff0e5fa020045d5da33f3778e23111639e870fbca962adcdb3e: Status 404 returned error can't find the container with id 14caf970d24ccff0e5fa020045d5da33f3778e23111639e870fbca962adcdb3e Aug 04 00:08:27.592963 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:27.592887 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:28.120248 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:28.120203 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" event={"ID":"6c4054c4-d6bf-4c47-96b0-aa6c818722f0","Type":"ContainerStarted","Data":"14caf970d24ccff0e5fa020045d5da33f3778e23111639e870fbca962adcdb3e"} Aug 04 00:08:28.548157 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:28.548111 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:28.609486 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:28.609451 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:31.836920 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:31.836883 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:08:31.837269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:31.836966 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:08:32.144080 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.144046 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" event={"ID":"6c4054c4-d6bf-4c47-96b0-aa6c818722f0","Type":"ContainerStarted","Data":"19d8a91ef15d1c679b519159366bb399a4fbddb28af71f08b23f74f97a7e082d"} Aug 04 00:08:32.144250 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.144109 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:32.146071 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.146038 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" event={"ID":"2326363c-2143-4dec-8592-e5fdd7ab6dac","Type":"ContainerStarted","Data":"15a69d244b2a8b7c4dd362792ad93c50be3c79e6d9b6d5b4f62b3bbe663f6df9"} Aug 04 00:08:32.146071 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.146062 2551 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-7j49z container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 00:08:32.146260 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.146104 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:32.146260 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.146111 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" podUID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 00:08:32.147856 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.147829 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" event={"ID":"e129f437-cc33-4360-b132-927ae8b12083","Type":"ContainerStarted","Data":"c265cb635f692a224522287ed72bd608710cf1b4740fce77ec4be81b54059006"} Aug 04 00:08:32.149558 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.149521 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" event={"ID":"bf451cea-1e4b-4100-9ebd-8dc323e00e92","Type":"ContainerStarted","Data":"a9191c1e4b1aa6c4e43a40ce3eabebb770cc7839af0d556251f67bea09ee8a5c"} Aug 04 00:08:32.149687 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.149672 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:32.169252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.169197 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" podStartSLOduration=1.4667159779999999 podStartE2EDuration="6.169181964s" totalImagesPullingTime="4.702465986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:08:32.166779765 +0000 UTC m=+378.047227328" watchObservedRunningTime="2026-08-04 00:08:32.169181964 +0000 UTC m=+378.049629501" Aug 04 00:08:32.189149 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.189091 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57f7cf7b5f-xspfp" podStartSLOduration=1.141711812 podStartE2EDuration="7.189072296s" totalImagesPullingTime="6.047360484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:08:32.188573592 +0000 UTC m=+378.069021133" watchObservedRunningTime="2026-08-04 00:08:32.189072296 +0000 UTC m=+378.069519836" Aug 04 00:08:32.220620 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.220251 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" podStartSLOduration=1.022014309 podStartE2EDuration="6.220232236s" totalImagesPullingTime="5.198217927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:08:32.218083317 +0000 UTC m=+378.098530856" watchObservedRunningTime="2026-08-04 00:08:32.220232236 +0000 UTC m=+378.100679777" Aug 04 00:08:32.246108 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.246053 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" podStartSLOduration=1.286403349 podStartE2EDuration="7.246034279s" totalImagesPullingTime="5.95963093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:08:32.244487614 +0000 UTC m=+378.124935165" watchObservedRunningTime="2026-08-04 00:08:32.246034279 +0000 UTC m=+378.126481822" Aug 04 00:08:32.668846 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.668781 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:32.770312 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:32.770271 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:33.132116 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.132084 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:33.154319 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.154286 2551 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-7j49z container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 00:08:33.154458 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.154349 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" podUID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 00:08:33.232248 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.232210 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:33.348494 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.348464 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:33.573053 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.572975 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:33.684241 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.684201 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:33.801498 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:33.801463 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:34.936500 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:34.936463 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:35.043230 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:35.043162 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:35.387903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:35.387868 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:35.512930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:35.512896 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:36.154940 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:36.154907 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:36.223271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:36.223235 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:36.311826 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:36.311763 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:36.449464 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:36.449369 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:37.042541 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.042504 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z"] Aug 04 00:08:37.042742 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.042714 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" podUID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" containerName="discovery" containerID="cri-o://19d8a91ef15d1c679b519159366bb399a4fbddb28af71f08b23f74f97a7e082d" gracePeriod=30 Aug 04 00:08:37.176558 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.176519 2551 generic.go:356] "Generic (PLEG): container finished" podID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" containerID="19d8a91ef15d1c679b519159366bb399a4fbddb28af71f08b23f74f97a7e082d" exitCode=0 Aug 04 00:08:37.176959 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.176564 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" event={"ID":"6c4054c4-d6bf-4c47-96b0-aa6c818722f0","Type":"ContainerDied","Data":"19d8a91ef15d1c679b519159366bb399a4fbddb28af71f08b23f74f97a7e082d"} Aug 04 00:08:37.296316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.296255 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:37.440339 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440308 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-ca-configmap\") pod \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " Aug 04 00:08:37.440339 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440348 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4k8v\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-kube-api-access-k4k8v\") pod \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " Aug 04 00:08:37.440529 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440376 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-cacerts\") pod \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " Aug 04 00:08:37.440529 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440434 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-token\") pod \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " Aug 04 00:08:37.440529 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440458 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-local-certs\") pod \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " Aug 04 00:08:37.440529 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440494 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-kubeconfig\") pod \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " Aug 04 00:08:37.440742 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440562 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-dns-cert\") pod \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\" (UID: \"6c4054c4-d6bf-4c47-96b0-aa6c818722f0\") " Aug 04 00:08:37.440844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.440812 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6c4054c4-d6bf-4c47-96b0-aa6c818722f0" (UID: "6c4054c4-d6bf-4c47-96b0-aa6c818722f0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:08:37.443381 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.443347 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-local-certs" (OuterVolumeSpecName: "local-certs") pod "6c4054c4-d6bf-4c47-96b0-aa6c818722f0" (UID: "6c4054c4-d6bf-4c47-96b0-aa6c818722f0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:08:37.443381 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.443370 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6c4054c4-d6bf-4c47-96b0-aa6c818722f0" (UID: "6c4054c4-d6bf-4c47-96b0-aa6c818722f0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:08:37.443522 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.443357 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-cacerts" (OuterVolumeSpecName: "cacerts") pod "6c4054c4-d6bf-4c47-96b0-aa6c818722f0" (UID: "6c4054c4-d6bf-4c47-96b0-aa6c818722f0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:08:37.443522 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.443357 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-kube-api-access-k4k8v" (OuterVolumeSpecName: "kube-api-access-k4k8v") pod "6c4054c4-d6bf-4c47-96b0-aa6c818722f0" (UID: "6c4054c4-d6bf-4c47-96b0-aa6c818722f0"). InnerVolumeSpecName "kube-api-access-k4k8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:08:37.443522 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.443357 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c4054c4-d6bf-4c47-96b0-aa6c818722f0" (UID: "6c4054c4-d6bf-4c47-96b0-aa6c818722f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:08:37.443522 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.443453 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6c4054c4-d6bf-4c47-96b0-aa6c818722f0" (UID: "6c4054c4-d6bf-4c47-96b0-aa6c818722f0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:08:37.541147 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.541109 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-token\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:08:37.541147 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.541138 2551 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-local-certs\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:08:37.541147 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.541147 2551 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-kubeconfig\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:08:37.541147 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.541156 2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-dns-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:08:37.541383 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.541165 2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-istio-csr-ca-configmap\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:08:37.541383 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.541183 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4k8v\" (UniqueName: \"kubernetes.io/projected/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-kube-api-access-k4k8v\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:08:37.541383 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:37.541191 2551 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6c4054c4-d6bf-4c47-96b0-aa6c818722f0-cacerts\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:08:38.182570 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:38.182532 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" event={"ID":"6c4054c4-d6bf-4c47-96b0-aa6c818722f0","Type":"ContainerDied","Data":"14caf970d24ccff0e5fa020045d5da33f3778e23111639e870fbca962adcdb3e"} Aug 04 00:08:38.182570 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:38.182563 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z" Aug 04 00:08:38.183059 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:38.182606 2551 scope.go:117] "RemoveContainer" containerID="19d8a91ef15d1c679b519159366bb399a4fbddb28af71f08b23f74f97a7e082d" Aug 04 00:08:38.225826 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:38.225784 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z"] Aug 04 00:08:38.228995 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:38.228972 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7j49z"] Aug 04 00:08:38.657613 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:38.657571 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" path="/var/lib/kubelet/pods/6c4054c4-d6bf-4c47-96b0-aa6c818722f0/volumes" Aug 04 00:08:43.157805 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:43.157762 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-npdcp" Aug 04 00:08:43.166211 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:43.166186 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2fvbv" Aug 04 00:08:55.401059 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:55.401014 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:55.456657 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:55.456629 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:55.820051 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:55.819964 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:08:55.920994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:08:55.920957 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:26.226971 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:26.226933 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:26.324375 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:26.324324 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:29.048517 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.048484 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh"] Aug 04 00:09:29.048903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.048769 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" containerName="discovery" Aug 04 00:09:29.048903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.048779 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" containerName="discovery" Aug 04 00:09:29.048903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.048851 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4054c4-d6bf-4c47-96b0-aa6c818722f0" containerName="discovery" Aug 04 00:09:29.053012 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.052995 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.055440 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.055415 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b7jgn\"" Aug 04 00:09:29.055579 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.055415 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 00:09:29.056396 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.056381 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 00:09:29.060443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.059988 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh"] Aug 04 00:09:29.145087 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.145051 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx"] Aug 04 00:09:29.148412 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.148387 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.157200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.157179 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx"] Aug 04 00:09:29.163111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.163091 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.163193 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.163121 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.163193 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.163151 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mvnl\" (UniqueName: \"kubernetes.io/projected/f7878257-9a95-4c30-a46a-c4570cdd5cdc-kube-api-access-8mvnl\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.256008 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.255976 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv"] Aug 04 00:09:29.259609 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.259592 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.263474 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263455 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.263538 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263484 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.263538 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263510 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.263607 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263540 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mvnl\" (UniqueName: \"kubernetes.io/projected/f7878257-9a95-4c30-a46a-c4570cdd5cdc-kube-api-access-8mvnl\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.263607 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263563 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzjgk\" (UniqueName: \"kubernetes.io/projected/81958852-5f35-4a4c-9d69-8ce373bc32ff-kube-api-access-fzjgk\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.263607 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263600 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.263845 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263827 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.263897 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.263873 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.267602 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.267577 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv"] Aug 04 00:09:29.272809 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.272770 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mvnl\" (UniqueName: \"kubernetes.io/projected/f7878257-9a95-4c30-a46a-c4570cdd5cdc-kube-api-access-8mvnl\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.347125 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.347064 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb"] Aug 04 00:09:29.353233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.353206 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.359284 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.359257 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb"] Aug 04 00:09:29.363218 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.363194 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:29.364469 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.364446 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.364556 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.364483 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.364556 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.364511 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.364670 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.364650 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.364711 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.364703 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjgk\" (UniqueName: \"kubernetes.io/projected/81958852-5f35-4a4c-9d69-8ce373bc32ff-kube-api-access-fzjgk\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.364750 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.364727 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpl5k\" (UniqueName: \"kubernetes.io/projected/7bd37cd0-8c17-4a54-8537-bd794c0f301f-kube-api-access-tpl5k\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.364859 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.364839 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.365032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.365014 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.375390 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.375366 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzjgk\" (UniqueName: \"kubernetes.io/projected/81958852-5f35-4a4c-9d69-8ce373bc32ff-kube-api-access-fzjgk\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.457251 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.457223 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:29.468730 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.468698 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpl5k\" (UniqueName: \"kubernetes.io/projected/7bd37cd0-8c17-4a54-8537-bd794c0f301f-kube-api-access-tpl5k\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.468986 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.468759 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.468986 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.468841 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.468986 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.468868 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.468986 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.468920 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwsmc\" (UniqueName: \"kubernetes.io/projected/5f42f085-450e-40f8-aa48-a8f6b15bbc27-kube-api-access-vwsmc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.468986 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.468961 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.469436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.469409 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.469567 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.469484 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.478999 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.478965 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpl5k\" (UniqueName: \"kubernetes.io/projected/7bd37cd0-8c17-4a54-8537-bd794c0f301f-kube-api-access-tpl5k\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.494373 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.494351 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh"] Aug 04 00:09:29.496783 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:29.496753 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7878257_9a95_4c30_a46a_c4570cdd5cdc.slice/crio-ce791918637618d5a606a6a163fa963ea99094d0fb7b317e135de20cf85c4780 WatchSource:0}: Error finding container ce791918637618d5a606a6a163fa963ea99094d0fb7b317e135de20cf85c4780: Status 404 returned error can't find the container with id ce791918637618d5a606a6a163fa963ea99094d0fb7b317e135de20cf85c4780 Aug 04 00:09:29.569723 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.569593 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:29.569851 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.569821 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwsmc\" (UniqueName: \"kubernetes.io/projected/5f42f085-450e-40f8-aa48-a8f6b15bbc27-kube-api-access-vwsmc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.570104 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.569941 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.570104 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.569985 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.570396 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.570370 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.570530 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.570507 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.578062 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.578041 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwsmc\" (UniqueName: \"kubernetes.io/projected/5f42f085-450e-40f8-aa48-a8f6b15bbc27-kube-api-access-vwsmc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.597860 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.597779 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx"] Aug 04 00:09:29.598767 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:29.598734 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81958852_5f35_4a4c_9d69_8ce373bc32ff.slice/crio-0cbf7e8cabba25ee0dded0d84e673b4beb4f26a1ceac03c0c75f2de50604ade2 WatchSource:0}: Error finding container 0cbf7e8cabba25ee0dded0d84e673b4beb4f26a1ceac03c0c75f2de50604ade2: Status 404 returned error can't find the container with id 0cbf7e8cabba25ee0dded0d84e673b4beb4f26a1ceac03c0c75f2de50604ade2 Aug 04 00:09:29.664105 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.664079 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:29.714391 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.714298 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv"] Aug 04 00:09:29.717060 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:29.717016 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd37cd0_8c17_4a54_8537_bd794c0f301f.slice/crio-89a754f4301ddb5dc753f2e7a195110cce5b0e91bb49d670efa91d696f5c6aa0 WatchSource:0}: Error finding container 89a754f4301ddb5dc753f2e7a195110cce5b0e91bb49d670efa91d696f5c6aa0: Status 404 returned error can't find the container with id 89a754f4301ddb5dc753f2e7a195110cce5b0e91bb49d670efa91d696f5c6aa0 Aug 04 00:09:29.802850 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:29.802825 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb"] Aug 04 00:09:29.813192 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:29.813166 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f42f085_450e_40f8_aa48_a8f6b15bbc27.slice/crio-23e756f7e275842305f32f12b2000f8c2f0220ed30ef9166b36705e04f548e6a WatchSource:0}: Error finding container 23e756f7e275842305f32f12b2000f8c2f0220ed30ef9166b36705e04f548e6a: Status 404 returned error can't find the container with id 23e756f7e275842305f32f12b2000f8c2f0220ed30ef9166b36705e04f548e6a Aug 04 00:09:30.421980 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.421942 2551 generic.go:356] "Generic (PLEG): container finished" podID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerID="6337f1b6e6ef7890deacaaf9a6da9a153c322a6f62762dc035cae55ff6c48774" exitCode=0 Aug 04 00:09:30.422420 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.422024 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" event={"ID":"81958852-5f35-4a4c-9d69-8ce373bc32ff","Type":"ContainerDied","Data":"6337f1b6e6ef7890deacaaf9a6da9a153c322a6f62762dc035cae55ff6c48774"} Aug 04 00:09:30.422420 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.422071 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" event={"ID":"81958852-5f35-4a4c-9d69-8ce373bc32ff","Type":"ContainerStarted","Data":"0cbf7e8cabba25ee0dded0d84e673b4beb4f26a1ceac03c0c75f2de50604ade2"} Aug 04 00:09:30.423685 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.423646 2551 generic.go:356] "Generic (PLEG): container finished" podID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerID="d764e124827cbe6ab529e71bc013a3d47fa79d6ffe3dbeb3b9cd3874e2cffec3" exitCode=0 Aug 04 00:09:30.423825 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.423728 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" event={"ID":"f7878257-9a95-4c30-a46a-c4570cdd5cdc","Type":"ContainerDied","Data":"d764e124827cbe6ab529e71bc013a3d47fa79d6ffe3dbeb3b9cd3874e2cffec3"} Aug 04 00:09:30.423825 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.423779 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" event={"ID":"f7878257-9a95-4c30-a46a-c4570cdd5cdc","Type":"ContainerStarted","Data":"ce791918637618d5a606a6a163fa963ea99094d0fb7b317e135de20cf85c4780"} Aug 04 00:09:30.425166 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.425143 2551 generic.go:356] "Generic (PLEG): container finished" podID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerID="e1ca8558cb0fb3724cc7bb7af908a2ef1e8ccc2e93990d0783eed1b1851886ae" exitCode=0 Aug 04 00:09:30.425249 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.425222 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" event={"ID":"5f42f085-450e-40f8-aa48-a8f6b15bbc27","Type":"ContainerDied","Data":"e1ca8558cb0fb3724cc7bb7af908a2ef1e8ccc2e93990d0783eed1b1851886ae"} Aug 04 00:09:30.425305 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.425258 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" event={"ID":"5f42f085-450e-40f8-aa48-a8f6b15bbc27","Type":"ContainerStarted","Data":"23e756f7e275842305f32f12b2000f8c2f0220ed30ef9166b36705e04f548e6a"} Aug 04 00:09:30.426884 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.426867 2551 generic.go:356] "Generic (PLEG): container finished" podID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerID="aef0c363048aca4892160b5dfd12496003bc2793529c42c694125d2db08a021c" exitCode=0 Aug 04 00:09:30.426966 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.426890 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" event={"ID":"7bd37cd0-8c17-4a54-8537-bd794c0f301f","Type":"ContainerDied","Data":"aef0c363048aca4892160b5dfd12496003bc2793529c42c694125d2db08a021c"} Aug 04 00:09:30.426966 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:30.426926 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" event={"ID":"7bd37cd0-8c17-4a54-8537-bd794c0f301f","Type":"ContainerStarted","Data":"89a754f4301ddb5dc753f2e7a195110cce5b0e91bb49d670efa91d696f5c6aa0"} Aug 04 00:09:32.438199 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:32.438170 2551 generic.go:356] "Generic (PLEG): container finished" podID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerID="311f9661c910da5790fa61b3a27c498138e8f7fb482448560be7ae5644b289ed" exitCode=0 Aug 04 00:09:32.438680 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:32.438255 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" event={"ID":"5f42f085-450e-40f8-aa48-a8f6b15bbc27","Type":"ContainerDied","Data":"311f9661c910da5790fa61b3a27c498138e8f7fb482448560be7ae5644b289ed"} Aug 04 00:09:32.445657 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:32.445621 2551 generic.go:356] "Generic (PLEG): container finished" podID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerID="d5f92bc2e3e0611fe16a3b90f5aebc4a4102aac088f67a50163200576fa6ddcf" exitCode=0 Aug 04 00:09:32.445770 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:32.445661 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" event={"ID":"7bd37cd0-8c17-4a54-8537-bd794c0f301f","Type":"ContainerDied","Data":"d5f92bc2e3e0611fe16a3b90f5aebc4a4102aac088f67a50163200576fa6ddcf"} Aug 04 00:09:33.452932 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.452897 2551 generic.go:356] "Generic (PLEG): container finished" podID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerID="e2d3e5739c06ec48cef1b084fe3d1c8514e9c04cd40aef2ac9941843d4f2dd3c" exitCode=0 Aug 04 00:09:33.453329 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.452990 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" event={"ID":"7bd37cd0-8c17-4a54-8537-bd794c0f301f","Type":"ContainerDied","Data":"e2d3e5739c06ec48cef1b084fe3d1c8514e9c04cd40aef2ac9941843d4f2dd3c"} Aug 04 00:09:33.454765 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.454740 2551 generic.go:356] "Generic (PLEG): container finished" podID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerID="b7e002061c9d236e9675a0ac132919e9ed215a5e83c86679e5d33c9bbe24fd89" exitCode=0 Aug 04 00:09:33.454915 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.454823 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" event={"ID":"81958852-5f35-4a4c-9d69-8ce373bc32ff","Type":"ContainerDied","Data":"b7e002061c9d236e9675a0ac132919e9ed215a5e83c86679e5d33c9bbe24fd89"} Aug 04 00:09:33.456558 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.456537 2551 generic.go:356] "Generic (PLEG): container finished" podID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerID="bb33de9f1c74a06a11f4afc68a661b29c4beb641377b42f4a57f0f8b6e56f4c6" exitCode=0 Aug 04 00:09:33.456649 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.456613 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" event={"ID":"f7878257-9a95-4c30-a46a-c4570cdd5cdc","Type":"ContainerDied","Data":"bb33de9f1c74a06a11f4afc68a661b29c4beb641377b42f4a57f0f8b6e56f4c6"} Aug 04 00:09:33.458691 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.458674 2551 generic.go:356] "Generic (PLEG): container finished" podID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerID="839573414d690316fdbc67903fde7eb33eed326ece427dd2a6a3650911d785ca" exitCode=0 Aug 04 00:09:33.458770 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:33.458699 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" event={"ID":"5f42f085-450e-40f8-aa48-a8f6b15bbc27","Type":"ContainerDied","Data":"839573414d690316fdbc67903fde7eb33eed326ece427dd2a6a3650911d785ca"} Aug 04 00:09:34.464510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.464477 2551 generic.go:356] "Generic (PLEG): container finished" podID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerID="c8f99bde537616ae94f9e3cf76b03181aa7cd5020104665e57999531f0016f8e" exitCode=0 Aug 04 00:09:34.464983 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.464546 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" event={"ID":"81958852-5f35-4a4c-9d69-8ce373bc32ff","Type":"ContainerDied","Data":"c8f99bde537616ae94f9e3cf76b03181aa7cd5020104665e57999531f0016f8e"} Aug 04 00:09:34.466492 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.466469 2551 generic.go:356] "Generic (PLEG): container finished" podID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerID="e627a7b56833b89d6a42b9c252e58c155aeb46f929afbe9f8fa9956056707fe0" exitCode=0 Aug 04 00:09:34.466613 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.466546 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" event={"ID":"f7878257-9a95-4c30-a46a-c4570cdd5cdc","Type":"ContainerDied","Data":"e627a7b56833b89d6a42b9c252e58c155aeb46f929afbe9f8fa9956056707fe0"} Aug 04 00:09:34.620618 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.620595 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:34.623405 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.623384 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:34.715279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715201 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwsmc\" (UniqueName: \"kubernetes.io/projected/5f42f085-450e-40f8-aa48-a8f6b15bbc27-kube-api-access-vwsmc\") pod \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " Aug 04 00:09:34.715279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715237 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-bundle\") pod \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " Aug 04 00:09:34.715279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715266 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-bundle\") pod \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " Aug 04 00:09:34.715525 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715355 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-util\") pod \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " Aug 04 00:09:34.715525 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715436 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpl5k\" (UniqueName: \"kubernetes.io/projected/7bd37cd0-8c17-4a54-8537-bd794c0f301f-kube-api-access-tpl5k\") pod \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\" (UID: \"7bd37cd0-8c17-4a54-8537-bd794c0f301f\") " Aug 04 00:09:34.715525 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715472 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-util\") pod \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\" (UID: \"5f42f085-450e-40f8-aa48-a8f6b15bbc27\") " Aug 04 00:09:34.715861 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715830 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-bundle" (OuterVolumeSpecName: "bundle") pod "5f42f085-450e-40f8-aa48-a8f6b15bbc27" (UID: "5f42f085-450e-40f8-aa48-a8f6b15bbc27"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:34.715998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.715957 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-bundle" (OuterVolumeSpecName: "bundle") pod "7bd37cd0-8c17-4a54-8537-bd794c0f301f" (UID: "7bd37cd0-8c17-4a54-8537-bd794c0f301f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:34.717334 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.717309 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f42f085-450e-40f8-aa48-a8f6b15bbc27-kube-api-access-vwsmc" (OuterVolumeSpecName: "kube-api-access-vwsmc") pod "5f42f085-450e-40f8-aa48-a8f6b15bbc27" (UID: "5f42f085-450e-40f8-aa48-a8f6b15bbc27"). InnerVolumeSpecName "kube-api-access-vwsmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:09:34.717623 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.717599 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd37cd0-8c17-4a54-8537-bd794c0f301f-kube-api-access-tpl5k" (OuterVolumeSpecName: "kube-api-access-tpl5k") pod "7bd37cd0-8c17-4a54-8537-bd794c0f301f" (UID: "7bd37cd0-8c17-4a54-8537-bd794c0f301f"). InnerVolumeSpecName "kube-api-access-tpl5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:09:34.721157 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.721123 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-util" (OuterVolumeSpecName: "util") pod "5f42f085-450e-40f8-aa48-a8f6b15bbc27" (UID: "5f42f085-450e-40f8-aa48-a8f6b15bbc27"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:34.723341 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.723317 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-util" (OuterVolumeSpecName: "util") pod "7bd37cd0-8c17-4a54-8537-bd794c0f301f" (UID: "7bd37cd0-8c17-4a54-8537-bd794c0f301f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:34.816333 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.816297 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-util\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:34.816485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.816347 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpl5k\" (UniqueName: \"kubernetes.io/projected/7bd37cd0-8c17-4a54-8537-bd794c0f301f-kube-api-access-tpl5k\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:34.816485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.816358 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-util\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:34.816485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.816367 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwsmc\" (UniqueName: \"kubernetes.io/projected/5f42f085-450e-40f8-aa48-a8f6b15bbc27-kube-api-access-vwsmc\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:34.816485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.816375 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f42f085-450e-40f8-aa48-a8f6b15bbc27-bundle\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:34.816485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:34.816384 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bd37cd0-8c17-4a54-8537-bd794c0f301f-bundle\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:35.474026 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.473999 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" Aug 04 00:09:35.474492 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.473996 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbcf5mb" event={"ID":"5f42f085-450e-40f8-aa48-a8f6b15bbc27","Type":"ContainerDied","Data":"23e756f7e275842305f32f12b2000f8c2f0220ed30ef9166b36705e04f548e6a"} Aug 04 00:09:35.474492 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.474137 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23e756f7e275842305f32f12b2000f8c2f0220ed30ef9166b36705e04f548e6a" Aug 04 00:09:35.475849 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.475822 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" event={"ID":"7bd37cd0-8c17-4a54-8537-bd794c0f301f","Type":"ContainerDied","Data":"89a754f4301ddb5dc753f2e7a195110cce5b0e91bb49d670efa91d696f5c6aa0"} Aug 04 00:09:35.475849 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.475844 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cgsvxv" Aug 04 00:09:35.476025 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.475874 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89a754f4301ddb5dc753f2e7a195110cce5b0e91bb49d670efa91d696f5c6aa0" Aug 04 00:09:35.611233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.611204 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:35.664570 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.664544 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:35.732573 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.732483 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-util\") pod \"81958852-5f35-4a4c-9d69-8ce373bc32ff\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " Aug 04 00:09:35.732573 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.732554 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzjgk\" (UniqueName: \"kubernetes.io/projected/81958852-5f35-4a4c-9d69-8ce373bc32ff-kube-api-access-fzjgk\") pod \"81958852-5f35-4a4c-9d69-8ce373bc32ff\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " Aug 04 00:09:35.732783 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.732579 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-bundle\") pod \"81958852-5f35-4a4c-9d69-8ce373bc32ff\" (UID: \"81958852-5f35-4a4c-9d69-8ce373bc32ff\") " Aug 04 00:09:35.733138 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.733108 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-bundle" (OuterVolumeSpecName: "bundle") pod "81958852-5f35-4a4c-9d69-8ce373bc32ff" (UID: "81958852-5f35-4a4c-9d69-8ce373bc32ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:35.734632 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.734612 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81958852-5f35-4a4c-9d69-8ce373bc32ff-kube-api-access-fzjgk" (OuterVolumeSpecName: "kube-api-access-fzjgk") pod "81958852-5f35-4a4c-9d69-8ce373bc32ff" (UID: "81958852-5f35-4a4c-9d69-8ce373bc32ff"). InnerVolumeSpecName "kube-api-access-fzjgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:09:35.737877 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.737843 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-util" (OuterVolumeSpecName: "util") pod "81958852-5f35-4a4c-9d69-8ce373bc32ff" (UID: "81958852-5f35-4a4c-9d69-8ce373bc32ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:35.833667 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.833630 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-bundle\") pod \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " Aug 04 00:09:35.833847 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.833707 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mvnl\" (UniqueName: \"kubernetes.io/projected/f7878257-9a95-4c30-a46a-c4570cdd5cdc-kube-api-access-8mvnl\") pod \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " Aug 04 00:09:35.833847 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.833747 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-util\") pod \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\" (UID: \"f7878257-9a95-4c30-a46a-c4570cdd5cdc\") " Aug 04 00:09:35.833938 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.833921 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-util\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:35.833975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.833945 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzjgk\" (UniqueName: \"kubernetes.io/projected/81958852-5f35-4a4c-9d69-8ce373bc32ff-kube-api-access-fzjgk\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:35.833975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.833961 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81958852-5f35-4a4c-9d69-8ce373bc32ff-bundle\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:35.834452 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.834420 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-bundle" (OuterVolumeSpecName: "bundle") pod "f7878257-9a95-4c30-a46a-c4570cdd5cdc" (UID: "f7878257-9a95-4c30-a46a-c4570cdd5cdc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:35.835778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.835754 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7878257-9a95-4c30-a46a-c4570cdd5cdc-kube-api-access-8mvnl" (OuterVolumeSpecName: "kube-api-access-8mvnl") pod "f7878257-9a95-4c30-a46a-c4570cdd5cdc" (UID: "f7878257-9a95-4c30-a46a-c4570cdd5cdc"). InnerVolumeSpecName "kube-api-access-8mvnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:09:35.839437 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.839415 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-util" (OuterVolumeSpecName: "util") pod "f7878257-9a95-4c30-a46a-c4570cdd5cdc" (UID: "f7878257-9a95-4c30-a46a-c4570cdd5cdc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:09:35.935322 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.935285 2551 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-util\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:35.935322 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.935315 2551 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7878257-9a95-4c30-a46a-c4570cdd5cdc-bundle\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:35.935322 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:35.935324 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mvnl\" (UniqueName: \"kubernetes.io/projected/f7878257-9a95-4c30-a46a-c4570cdd5cdc-kube-api-access-8mvnl\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:09:36.482585 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:36.482554 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" event={"ID":"81958852-5f35-4a4c-9d69-8ce373bc32ff","Type":"ContainerDied","Data":"0cbf7e8cabba25ee0dded0d84e673b4beb4f26a1ceac03c0c75f2de50604ade2"} Aug 04 00:09:36.483040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:36.482604 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cbf7e8cabba25ee0dded0d84e673b4beb4f26a1ceac03c0c75f2de50604ade2" Aug 04 00:09:36.483040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:36.482561 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0p64hx" Aug 04 00:09:36.484416 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:36.484391 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" event={"ID":"f7878257-9a95-4c30-a46a-c4570cdd5cdc","Type":"ContainerDied","Data":"ce791918637618d5a606a6a163fa963ea99094d0fb7b317e135de20cf85c4780"} Aug 04 00:09:36.484527 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:36.484420 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053gptgh" Aug 04 00:09:36.484527 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:36.484429 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce791918637618d5a606a6a163fa963ea99094d0fb7b317e135de20cf85c4780" Aug 04 00:09:42.001463 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:42.001426 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:42.987900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:42.987865 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:45.856393 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856363 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-l5jrb"] Aug 04 00:09:45.856844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856823 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerName="pull" Aug 04 00:09:45.856844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856842 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerName="pull" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856857 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerName="extract" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856866 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerName="extract" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856879 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerName="pull" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856886 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerName="pull" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856895 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerName="extract" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856903 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerName="extract" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856914 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerName="util" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856922 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerName="util" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856931 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerName="pull" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856939 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerName="pull" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856953 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerName="util" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856964 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerName="util" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856974 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerName="extract" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856982 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerName="extract" Aug 04 00:09:45.856989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.856993 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerName="util" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857001 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerName="util" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857018 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerName="util" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857026 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerName="util" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857035 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerName="pull" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857042 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerName="pull" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857056 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerName="extract" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857064 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerName="extract" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857140 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7878257-9a95-4c30-a46a-c4570cdd5cdc" containerName="extract" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857156 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bd37cd0-8c17-4a54-8537-bd794c0f301f" containerName="extract" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857166 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f42f085-450e-40f8-aa48-a8f6b15bbc27" containerName="extract" Aug 04 00:09:45.857644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.857176 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="81958852-5f35-4a4c-9d69-8ce373bc32ff" containerName="extract" Aug 04 00:09:45.860247 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.860227 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" Aug 04 00:09:45.862759 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.862740 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 00:09:45.862873 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.862780 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 00:09:45.863114 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.863097 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4kwtf\"" Aug 04 00:09:45.870492 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:45.870471 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-l5jrb"] Aug 04 00:09:46.016956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.016926 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb9qw\" (UniqueName: \"kubernetes.io/projected/93920fe4-ea33-49e7-965b-1dcfbebe35e8-kube-api-access-jb9qw\") pod \"authorino-operator-68d88948d8-l5jrb\" (UID: \"93920fe4-ea33-49e7-965b-1dcfbebe35e8\") " pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" Aug 04 00:09:46.117921 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.117827 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb9qw\" (UniqueName: \"kubernetes.io/projected/93920fe4-ea33-49e7-965b-1dcfbebe35e8-kube-api-access-jb9qw\") pod \"authorino-operator-68d88948d8-l5jrb\" (UID: \"93920fe4-ea33-49e7-965b-1dcfbebe35e8\") " pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" Aug 04 00:09:46.132409 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.132376 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb9qw\" (UniqueName: \"kubernetes.io/projected/93920fe4-ea33-49e7-965b-1dcfbebe35e8-kube-api-access-jb9qw\") pod \"authorino-operator-68d88948d8-l5jrb\" (UID: \"93920fe4-ea33-49e7-965b-1dcfbebe35e8\") " pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" Aug 04 00:09:46.170743 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.170715 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" Aug 04 00:09:46.300301 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.300276 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-l5jrb"] Aug 04 00:09:46.302367 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:46.302342 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93920fe4_ea33_49e7_965b_1dcfbebe35e8.slice/crio-a5c6f335afe8e650a7c615cceee79cb615f890390c8172983df77b07fabe12ae WatchSource:0}: Error finding container a5c6f335afe8e650a7c615cceee79cb615f890390c8172983df77b07fabe12ae: Status 404 returned error can't find the container with id a5c6f335afe8e650a7c615cceee79cb615f890390c8172983df77b07fabe12ae Aug 04 00:09:46.408292 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.408259 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:46.530964 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.530931 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" event={"ID":"93920fe4-ea33-49e7-965b-1dcfbebe35e8","Type":"ContainerStarted","Data":"a5c6f335afe8e650a7c615cceee79cb615f890390c8172983df77b07fabe12ae"} Aug 04 00:09:46.628446 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.628412 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:46.797094 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:46.797013 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:47.395928 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:47.395890 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:49.550065 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:49.550028 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" event={"ID":"93920fe4-ea33-49e7-965b-1dcfbebe35e8","Type":"ContainerStarted","Data":"702edf5663b015a1eb6a0d967b57a5fb1606540ea3174218774d522ee63866eb"} Aug 04 00:09:49.550452 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:49.550093 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" Aug 04 00:09:49.569889 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:49.569838 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" podStartSLOduration=2.005063682 podStartE2EDuration="4.569820988s" totalImagesPullingTime="2.564757306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:09:49.567509738 +0000 UTC m=+455.447957296" watchObservedRunningTime="2026-08-04 00:09:49.569820988 +0000 UTC m=+455.450268525" Aug 04 00:09:50.121545 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:50.121509 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:50.209700 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:50.209664 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:51.152556 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.152523 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26"] Aug 04 00:09:51.155876 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.155857 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.158583 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.158558 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mgscw\"" Aug 04 00:09:51.170113 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.170090 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26"] Aug 04 00:09:51.257349 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.257315 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8542f660-bfa9-4446-8cdd-4e0085699d8e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.257349 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.257347 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srbhm\" (UniqueName: \"kubernetes.io/projected/8542f660-bfa9-4446-8cdd-4e0085699d8e-kube-api-access-srbhm\") pod \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.358113 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.358076 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8542f660-bfa9-4446-8cdd-4e0085699d8e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.358259 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.358115 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srbhm\" (UniqueName: \"kubernetes.io/projected/8542f660-bfa9-4446-8cdd-4e0085699d8e-kube-api-access-srbhm\") pod \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.358458 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.358435 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8542f660-bfa9-4446-8cdd-4e0085699d8e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.366173 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.366150 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srbhm\" (UniqueName: \"kubernetes.io/projected/8542f660-bfa9-4446-8cdd-4e0085699d8e-kube-api-access-srbhm\") pod \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.465779 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.465703 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:51.599036 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:51.599013 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26"] Aug 04 00:09:51.600599 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:51.600573 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8542f660_bfa9_4446_8cdd_4e0085699d8e.slice/crio-d58a6506823eb6c1bd89e45217caecda4bb1ec335b6a45cd64291f3d2c9a06be WatchSource:0}: Error finding container d58a6506823eb6c1bd89e45217caecda4bb1ec335b6a45cd64291f3d2c9a06be: Status 404 returned error can't find the container with id d58a6506823eb6c1bd89e45217caecda4bb1ec335b6a45cd64291f3d2c9a06be Aug 04 00:09:52.270882 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:52.270789 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:52.568697 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:52.568610 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" event={"ID":"8542f660-bfa9-4446-8cdd-4e0085699d8e","Type":"ContainerStarted","Data":"d58a6506823eb6c1bd89e45217caecda4bb1ec335b6a45cd64291f3d2c9a06be"} Aug 04 00:09:54.233960 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.233924 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv"] Aug 04 00:09:54.237228 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.237207 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" Aug 04 00:09:54.239710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.239691 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Aug 04 00:09:54.239930 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.239916 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-gk9ln\"" Aug 04 00:09:54.248907 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.248887 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv"] Aug 04 00:09:54.282840 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.282809 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcjqn\" (UniqueName: \"kubernetes.io/projected/c6331656-1633-4dec-971b-aac2fec661f0-kube-api-access-zcjqn\") pod \"dns-operator-controller-manager-5985564776-x9qlv\" (UID: \"c6331656-1633-4dec-971b-aac2fec661f0\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" Aug 04 00:09:54.384167 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.384133 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcjqn\" (UniqueName: \"kubernetes.io/projected/c6331656-1633-4dec-971b-aac2fec661f0-kube-api-access-zcjqn\") pod \"dns-operator-controller-manager-5985564776-x9qlv\" (UID: \"c6331656-1633-4dec-971b-aac2fec661f0\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" Aug 04 00:09:54.397011 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.396986 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcjqn\" (UniqueName: \"kubernetes.io/projected/c6331656-1633-4dec-971b-aac2fec661f0-kube-api-access-zcjqn\") pod \"dns-operator-controller-manager-5985564776-x9qlv\" (UID: \"c6331656-1633-4dec-971b-aac2fec661f0\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" Aug 04 00:09:54.548883 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.548776 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" Aug 04 00:09:54.686191 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:54.686154 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv"] Aug 04 00:09:54.690781 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:54.690750 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6331656_1633_4dec_971b_aac2fec661f0.slice/crio-290db00d67eb17170907ac040fde184deead9acb98c7b3642360db228ea4da52 WatchSource:0}: Error finding container 290db00d67eb17170907ac040fde184deead9acb98c7b3642360db228ea4da52: Status 404 returned error can't find the container with id 290db00d67eb17170907ac040fde184deead9acb98c7b3642360db228ea4da52 Aug 04 00:09:55.197990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:55.197919 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:55.585589 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:55.585510 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" event={"ID":"c6331656-1633-4dec-971b-aac2fec661f0","Type":"ContainerStarted","Data":"290db00d67eb17170907ac040fde184deead9acb98c7b3642360db228ea4da52"} Aug 04 00:09:57.240519 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.240483 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8"] Aug 04 00:09:57.244201 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.244176 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:09:57.246702 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.246674 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-p8qt6\"" Aug 04 00:09:57.252677 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.252651 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8"] Aug 04 00:09:57.309749 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.309714 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8tvn\" (UniqueName: \"kubernetes.io/projected/ec3b52da-0928-408b-9253-9133b8b6ce77-kube-api-access-l8tvn\") pod \"limitador-operator-controller-manager-6c549b6d85-fllx8\" (UID: \"ec3b52da-0928-408b-9253-9133b8b6ce77\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:09:57.410222 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.410196 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8tvn\" (UniqueName: \"kubernetes.io/projected/ec3b52da-0928-408b-9253-9133b8b6ce77-kube-api-access-l8tvn\") pod \"limitador-operator-controller-manager-6c549b6d85-fllx8\" (UID: \"ec3b52da-0928-408b-9253-9133b8b6ce77\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:09:57.423831 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.423809 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8tvn\" (UniqueName: \"kubernetes.io/projected/ec3b52da-0928-408b-9253-9133b8b6ce77-kube-api-access-l8tvn\") pod \"limitador-operator-controller-manager-6c549b6d85-fllx8\" (UID: \"ec3b52da-0928-408b-9253-9133b8b6ce77\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:09:57.557998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.557916 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:09:57.598317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.598272 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" event={"ID":"8542f660-bfa9-4446-8cdd-4e0085699d8e","Type":"ContainerStarted","Data":"a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281"} Aug 04 00:09:57.598504 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.598427 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:09:57.600083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.600055 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" event={"ID":"c6331656-1633-4dec-971b-aac2fec661f0","Type":"ContainerStarted","Data":"055ceb3112ed383ad19124f379f937f9dcd71cd13d2460c5948a62318cbfd8fe"} Aug 04 00:09:57.600201 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.600187 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" Aug 04 00:09:57.620944 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.620787 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" podStartSLOduration=0.819541636 podStartE2EDuration="6.620765922s" totalImagesPullingTime="5.801224286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:09:57.618987873 +0000 UTC m=+463.499435414" watchObservedRunningTime="2026-08-04 00:09:57.620765922 +0000 UTC m=+463.501213466" Aug 04 00:09:57.641639 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.641577 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" podStartSLOduration=0.93732187 podStartE2EDuration="3.64155733s" totalImagesPullingTime="2.70423546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:09:57.638900685 +0000 UTC m=+463.519348221" watchObservedRunningTime="2026-08-04 00:09:57.64155733 +0000 UTC m=+463.522004872" Aug 04 00:09:57.698618 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:57.698587 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8"] Aug 04 00:09:57.699965 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:09:57.699936 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3b52da_0928_408b_9253_9133b8b6ce77.slice/crio-fe0a06965b78dbb34038d0aa17e5c6c4e7ce92d67bcf294cb84e483791e9098c WatchSource:0}: Error finding container fe0a06965b78dbb34038d0aa17e5c6c4e7ce92d67bcf294cb84e483791e9098c: Status 404 returned error can't find the container with id fe0a06965b78dbb34038d0aa17e5c6c4e7ce92d67bcf294cb84e483791e9098c Aug 04 00:09:58.606621 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:58.606585 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" event={"ID":"ec3b52da-0928-408b-9253-9133b8b6ce77","Type":"ContainerStarted","Data":"fe0a06965b78dbb34038d0aa17e5c6c4e7ce92d67bcf294cb84e483791e9098c"} Aug 04 00:09:59.195693 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:59.195660 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:59.264741 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:59.264704 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:59.805103 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:59.805024 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:09:59.874235 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:09:59.874193 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:00.293205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.293165 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh"] Aug 04 00:10:00.297107 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.297065 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.298564 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.298537 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:00.299673 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.299655 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Aug 04 00:10:00.299760 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.299672 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sl686\"" Aug 04 00:10:00.299760 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.299657 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Aug 04 00:10:00.304662 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.304552 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh"] Aug 04 00:10:00.338653 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.338568 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d969fb-3209-4e74-87c9-713ec1c35cc1-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.338653 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.338608 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w58d\" (UniqueName: \"kubernetes.io/projected/c0d969fb-3209-4e74-87c9-713ec1c35cc1-kube-api-access-8w58d\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.338854 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.338658 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0d969fb-3209-4e74-87c9-713ec1c35cc1-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.439570 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.439535 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0d969fb-3209-4e74-87c9-713ec1c35cc1-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.439766 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.439599 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d969fb-3209-4e74-87c9-713ec1c35cc1-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.439766 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.439621 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w58d\" (UniqueName: \"kubernetes.io/projected/c0d969fb-3209-4e74-87c9-713ec1c35cc1-kube-api-access-8w58d\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.439766 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:00.439743 2551 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Aug 04 00:10:00.439953 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:00.439834 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d969fb-3209-4e74-87c9-713ec1c35cc1-plugin-serving-cert podName:c0d969fb-3209-4e74-87c9-713ec1c35cc1 nodeName:}" failed. No retries permitted until 2026-08-04 00:10:00.939812406 +0000 UTC m=+466.820259924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c0d969fb-3209-4e74-87c9-713ec1c35cc1-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-jc6zh" (UID: "c0d969fb-3209-4e74-87c9-713ec1c35cc1") : secret "plugin-serving-cert" not found Aug 04 00:10:00.440292 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.440267 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0d969fb-3209-4e74-87c9-713ec1c35cc1-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.447814 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.447782 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w58d\" (UniqueName: \"kubernetes.io/projected/c0d969fb-3209-4e74-87c9-713ec1c35cc1-kube-api-access-8w58d\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.557869 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.557840 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-l5jrb" Aug 04 00:10:00.621277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.621202 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" event={"ID":"ec3b52da-0928-408b-9253-9133b8b6ce77","Type":"ContainerStarted","Data":"62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214"} Aug 04 00:10:00.621406 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.621306 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:00.639564 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.639519 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" podStartSLOduration=1.336436451 podStartE2EDuration="3.639503556s" totalImagesPullingTime="2.303067105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:00.637720641 +0000 UTC m=+466.518168180" watchObservedRunningTime="2026-08-04 00:10:00.639503556 +0000 UTC m=+466.519951095" Aug 04 00:10:00.719960 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.719927 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:00.943991 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.943952 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d969fb-3209-4e74-87c9-713ec1c35cc1-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:00.946423 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:00.946399 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d969fb-3209-4e74-87c9-713ec1c35cc1-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-jc6zh\" (UID: \"c0d969fb-3209-4e74-87c9-713ec1c35cc1\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:01.208491 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:01.208409 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" Aug 04 00:10:01.338075 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:01.338047 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh"] Aug 04 00:10:01.339597 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:01.339574 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0d969fb_3209_4e74_87c9_713ec1c35cc1.slice/crio-49c05366134f1617d5e9f2d9a8d0c36c6b74b4dea1fae51ea2ad9b1e5531d888 WatchSource:0}: Error finding container 49c05366134f1617d5e9f2d9a8d0c36c6b74b4dea1fae51ea2ad9b1e5531d888: Status 404 returned error can't find the container with id 49c05366134f1617d5e9f2d9a8d0c36c6b74b4dea1fae51ea2ad9b1e5531d888 Aug 04 00:10:01.627830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:01.627787 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" event={"ID":"c0d969fb-3209-4e74-87c9-713ec1c35cc1","Type":"ContainerStarted","Data":"49c05366134f1617d5e9f2d9a8d0c36c6b74b4dea1fae51ea2ad9b1e5531d888"} Aug 04 00:10:05.199761 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:05.199724 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:05.299751 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:05.299714 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:05.796995 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:05.796960 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:05.898762 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:05.898715 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:06.396517 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:06.396478 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:06.458850 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:06.458788 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:07.663941 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:07.663909 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" event={"ID":"c0d969fb-3209-4e74-87c9-713ec1c35cc1","Type":"ContainerStarted","Data":"4beadf6f2318427bdc5bcbe6c19759f042de375b2fad50d92870d78299efccd2"} Aug 04 00:10:07.681450 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:07.681403 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-jc6zh" podStartSLOduration=2.152465738 podStartE2EDuration="7.681388803s" totalImagesPullingTime="5.528923065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:07.680063888 +0000 UTC m=+473.560511429" watchObservedRunningTime="2026-08-04 00:10:07.681388803 +0000 UTC m=+473.561836344" Aug 04 00:10:08.608557 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:08.608524 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-x9qlv" Aug 04 00:10:08.608720 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:08.608598 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.276599 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.276562 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26"] Aug 04 00:10:10.277006 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.276755 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" containerName="manager" containerID="cri-o://a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281" gracePeriod=2 Aug 04 00:10:10.278945 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.278914 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.281448 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.281423 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26"] Aug 04 00:10:10.292096 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.292075 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8"] Aug 04 00:10:10.295848 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.292694 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" containerName="manager" containerID="cri-o://62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214" gracePeriod=2 Aug 04 00:10:10.298669 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.297325 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.301229 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.301104 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.306409 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.306382 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj"] Aug 04 00:10:10.307465 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.306885 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" containerName="manager" Aug 04 00:10:10.307465 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.306907 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" containerName="manager" Aug 04 00:10:10.307465 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.307002 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" containerName="manager" Aug 04 00:10:10.311242 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.311210 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8"] Aug 04 00:10:10.311354 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.311326 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.317552 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.317513 2551 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ec3b52da-0928-408b-9253-9133b8b6ce77\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-fllx8\": pods \"limitador-operator-controller-manager-6c549b6d85-fllx8\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.320698 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.320356 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2"] Aug 04 00:10:10.320818 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.320777 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" containerName="manager" Aug 04 00:10:10.320818 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.320810 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" containerName="manager" Aug 04 00:10:10.320944 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.320935 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" containerName="manager" Aug 04 00:10:10.321044 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.321011 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.323944 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.323923 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj"] Aug 04 00:10:10.324039 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.324027 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" Aug 04 00:10:10.328520 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.328494 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-fllx8\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.335934 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.335912 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2"] Aug 04 00:10:10.352655 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.352631 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.354663 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.354632 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-fllx8\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.430127 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.430102 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-82jsj\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.430261 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.430165 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vxqr\" (UniqueName: \"kubernetes.io/projected/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-kube-api-access-6vxqr\") pod \"kuadrant-operator-controller-manager-58fc56757c-82jsj\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.430261 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.430186 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gpdk\" (UniqueName: \"kubernetes.io/projected/7c8667d8-9b61-43a5-8e6a-2eedd5372281-kube-api-access-8gpdk\") pod \"limitador-operator-controller-manager-6c549b6d85-tvzb2\" (UID: \"7c8667d8-9b61-43a5-8e6a-2eedd5372281\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" Aug 04 00:10:10.530751 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.530717 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-82jsj\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.530866 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.530821 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vxqr\" (UniqueName: \"kubernetes.io/projected/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-kube-api-access-6vxqr\") pod \"kuadrant-operator-controller-manager-58fc56757c-82jsj\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.530866 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.530848 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gpdk\" (UniqueName: \"kubernetes.io/projected/7c8667d8-9b61-43a5-8e6a-2eedd5372281-kube-api-access-8gpdk\") pod \"limitador-operator-controller-manager-6c549b6d85-tvzb2\" (UID: \"7c8667d8-9b61-43a5-8e6a-2eedd5372281\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" Aug 04 00:10:10.531075 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.531054 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-82jsj\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.542578 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.542557 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vxqr\" (UniqueName: \"kubernetes.io/projected/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-kube-api-access-6vxqr\") pod \"kuadrant-operator-controller-manager-58fc56757c-82jsj\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.542873 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.542853 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gpdk\" (UniqueName: \"kubernetes.io/projected/7c8667d8-9b61-43a5-8e6a-2eedd5372281-kube-api-access-8gpdk\") pod \"limitador-operator-controller-manager-6c549b6d85-tvzb2\" (UID: \"7c8667d8-9b61-43a5-8e6a-2eedd5372281\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" Aug 04 00:10:10.548432 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.548413 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.550428 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.550406 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.550495 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.550415 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.552455 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.552433 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-fllx8\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.554368 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.554347 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-x5k26\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.556085 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.556064 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-fllx8\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.631990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.631957 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8542f660-bfa9-4446-8cdd-4e0085699d8e-extensions-socket-volume\") pod \"8542f660-bfa9-4446-8cdd-4e0085699d8e\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " Aug 04 00:10:10.631990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.631990 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8tvn\" (UniqueName: \"kubernetes.io/projected/ec3b52da-0928-408b-9253-9133b8b6ce77-kube-api-access-l8tvn\") pod \"ec3b52da-0928-408b-9253-9133b8b6ce77\" (UID: \"ec3b52da-0928-408b-9253-9133b8b6ce77\") " Aug 04 00:10:10.632190 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.632054 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srbhm\" (UniqueName: \"kubernetes.io/projected/8542f660-bfa9-4446-8cdd-4e0085699d8e-kube-api-access-srbhm\") pod \"8542f660-bfa9-4446-8cdd-4e0085699d8e\" (UID: \"8542f660-bfa9-4446-8cdd-4e0085699d8e\") " Aug 04 00:10:10.632465 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.632429 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8542f660-bfa9-4446-8cdd-4e0085699d8e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8542f660-bfa9-4446-8cdd-4e0085699d8e" (UID: "8542f660-bfa9-4446-8cdd-4e0085699d8e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:10:10.634164 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.634141 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8542f660-bfa9-4446-8cdd-4e0085699d8e-kube-api-access-srbhm" (OuterVolumeSpecName: "kube-api-access-srbhm") pod "8542f660-bfa9-4446-8cdd-4e0085699d8e" (UID: "8542f660-bfa9-4446-8cdd-4e0085699d8e"). InnerVolumeSpecName "kube-api-access-srbhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:10:10.634164 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.634150 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3b52da-0928-408b-9253-9133b8b6ce77-kube-api-access-l8tvn" (OuterVolumeSpecName: "kube-api-access-l8tvn") pod "ec3b52da-0928-408b-9253-9133b8b6ce77" (UID: "ec3b52da-0928-408b-9253-9133b8b6ce77"). InnerVolumeSpecName "kube-api-access-l8tvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:10:10.658581 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.658549 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8542f660-bfa9-4446-8cdd-4e0085699d8e" path="/var/lib/kubelet/pods/8542f660-bfa9-4446-8cdd-4e0085699d8e/volumes" Aug 04 00:10:10.658941 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.658924 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec3b52da-0928-408b-9253-9133b8b6ce77" path="/var/lib/kubelet/pods/ec3b52da-0928-408b-9253-9133b8b6ce77/volumes" Aug 04 00:10:10.679332 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.679313 2551 generic.go:356] "Generic (PLEG): container finished" podID="ec3b52da-0928-408b-9253-9133b8b6ce77" containerID="62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214" exitCode=0 Aug 04 00:10:10.679426 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.679351 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fllx8" Aug 04 00:10:10.679426 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.679403 2551 scope.go:117] "RemoveContainer" containerID="62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214" Aug 04 00:10:10.681492 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.681473 2551 generic.go:356] "Generic (PLEG): container finished" podID="8542f660-bfa9-4446-8cdd-4e0085699d8e" containerID="a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281" exitCode=0 Aug 04 00:10:10.681585 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.681531 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-x5k26" Aug 04 00:10:10.687753 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.687738 2551 scope.go:117] "RemoveContainer" containerID="62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214" Aug 04 00:10:10.688037 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.688016 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214\": container with ID starting with 62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214 not found: ID does not exist" containerID="62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214" Aug 04 00:10:10.688097 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.688044 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214"} err="failed to get container status \"62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214\": rpc error: code = NotFound desc = could not find container \"62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214\": container with ID starting with 62be5cad9b571ba6cf224b4722a13f6b2ee64e6e7322ae60d5fc7706d9cef214 not found: ID does not exist" Aug 04 00:10:10.688097 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.688074 2551 scope.go:117] "RemoveContainer" containerID="a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281" Aug 04 00:10:10.695933 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.695914 2551 scope.go:117] "RemoveContainer" containerID="a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281" Aug 04 00:10:10.696173 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:10.696155 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281\": container with ID starting with a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281 not found: ID does not exist" containerID="a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281" Aug 04 00:10:10.696220 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.696183 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281"} err="failed to get container status \"a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281\": rpc error: code = NotFound desc = could not find container \"a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281\": container with ID starting with a82073716b30e47af9c2903d27f8e5391f1c01c896f383ee377481c368836281 not found: ID does not exist" Aug 04 00:10:10.722807 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.722775 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:10.729710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.729693 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" Aug 04 00:10:10.732542 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.732509 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srbhm\" (UniqueName: \"kubernetes.io/projected/8542f660-bfa9-4446-8cdd-4e0085699d8e-kube-api-access-srbhm\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:10:10.732542 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.732532 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8542f660-bfa9-4446-8cdd-4e0085699d8e-extensions-socket-volume\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:10:10.732626 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.732547 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8tvn\" (UniqueName: \"kubernetes.io/projected/ec3b52da-0928-408b-9253-9133b8b6ce77-kube-api-access-l8tvn\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:10:10.865055 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.865023 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj"] Aug 04 00:10:10.866843 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:10.866787 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57c06c6_fc0b_4968_89b1_811cd60f2b9f.slice/crio-93c5a001d67afd3cd2c107739868c3b2916c846a990d16f38a6761c01c8904a6 WatchSource:0}: Error finding container 93c5a001d67afd3cd2c107739868c3b2916c846a990d16f38a6761c01c8904a6: Status 404 returned error can't find the container with id 93c5a001d67afd3cd2c107739868c3b2916c846a990d16f38a6761c01c8904a6 Aug 04 00:10:10.879082 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:10.879060 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2"] Aug 04 00:10:10.883017 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:10.882992 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c8667d8_9b61_43a5_8e6a_2eedd5372281.slice/crio-90ea9fbdd5d1ea264985326b5ee3f32925ce977310e44d62d66a31c3ca08503a WatchSource:0}: Error finding container 90ea9fbdd5d1ea264985326b5ee3f32925ce977310e44d62d66a31c3ca08503a: Status 404 returned error can't find the container with id 90ea9fbdd5d1ea264985326b5ee3f32925ce977310e44d62d66a31c3ca08503a Aug 04 00:10:11.002888 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.002855 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl"] Aug 04 00:10:11.006151 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.006134 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.019840 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.019816 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl"] Aug 04 00:10:11.135541 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.135511 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp8nb\" (UniqueName: \"kubernetes.io/projected/e7c636de-b22b-4a68-95d6-37a448ccd073-kube-api-access-fp8nb\") pod \"kuadrant-operator-controller-manager-68cf589589-wxwjl\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.135693 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.135564 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c636de-b22b-4a68-95d6-37a448ccd073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-wxwjl\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.236499 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.236459 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c636de-b22b-4a68-95d6-37a448ccd073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-wxwjl\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.236674 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.236545 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp8nb\" (UniqueName: \"kubernetes.io/projected/e7c636de-b22b-4a68-95d6-37a448ccd073-kube-api-access-fp8nb\") pod \"kuadrant-operator-controller-manager-68cf589589-wxwjl\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.236895 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.236866 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c636de-b22b-4a68-95d6-37a448ccd073-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-wxwjl\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.244257 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.244239 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp8nb\" (UniqueName: \"kubernetes.io/projected/e7c636de-b22b-4a68-95d6-37a448ccd073-kube-api-access-fp8nb\") pod \"kuadrant-operator-controller-manager-68cf589589-wxwjl\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.315993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.315964 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.453744 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.453720 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl"] Aug 04 00:10:11.455827 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:11.455758 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c636de_b22b_4a68_95d6_37a448ccd073.slice/crio-a3bfddeec9321443c3a7cd13faaf9c97d1676099f76447a68f7aa3df31e2612e WatchSource:0}: Error finding container a3bfddeec9321443c3a7cd13faaf9c97d1676099f76447a68f7aa3df31e2612e: Status 404 returned error can't find the container with id a3bfddeec9321443c3a7cd13faaf9c97d1676099f76447a68f7aa3df31e2612e Aug 04 00:10:11.689176 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.689078 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" event={"ID":"e7c636de-b22b-4a68-95d6-37a448ccd073","Type":"ContainerStarted","Data":"7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8"} Aug 04 00:10:11.689176 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.689147 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" event={"ID":"e7c636de-b22b-4a68-95d6-37a448ccd073","Type":"ContainerStarted","Data":"a3bfddeec9321443c3a7cd13faaf9c97d1676099f76447a68f7aa3df31e2612e"} Aug 04 00:10:11.689402 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.689262 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:11.692068 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.692033 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" event={"ID":"f57c06c6-fc0b-4968-89b1-811cd60f2b9f","Type":"ContainerStarted","Data":"6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f"} Aug 04 00:10:11.692210 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.692093 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" event={"ID":"f57c06c6-fc0b-4968-89b1-811cd60f2b9f","Type":"ContainerStarted","Data":"93c5a001d67afd3cd2c107739868c3b2916c846a990d16f38a6761c01c8904a6"} Aug 04 00:10:11.692210 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.692137 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:11.695124 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.695099 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" event={"ID":"7c8667d8-9b61-43a5-8e6a-2eedd5372281","Type":"ContainerStarted","Data":"c622926ae458c43b248b251ff39219bf4bbf38e1ca70230a33022a7ccce30d5d"} Aug 04 00:10:11.695215 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.695139 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" event={"ID":"7c8667d8-9b61-43a5-8e6a-2eedd5372281","Type":"ContainerStarted","Data":"90ea9fbdd5d1ea264985326b5ee3f32925ce977310e44d62d66a31c3ca08503a"} Aug 04 00:10:11.695215 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.695172 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" Aug 04 00:10:11.718875 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.718831 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" podStartSLOduration=1.718814211 podStartE2EDuration="1.718814211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:11.716915366 +0000 UTC m=+477.597362905" watchObservedRunningTime="2026-08-04 00:10:11.718814211 +0000 UTC m=+477.599261748" Aug 04 00:10:11.740126 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.740080 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" podStartSLOduration=1.7400680099999999 podStartE2EDuration="1.74006801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:11.737229691 +0000 UTC m=+477.617677226" watchObservedRunningTime="2026-08-04 00:10:11.74006801 +0000 UTC m=+477.620515550" Aug 04 00:10:11.759594 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:11.759555 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" podStartSLOduration=1.759542326 podStartE2EDuration="1.759542326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:11.757684018 +0000 UTC m=+477.638131558" watchObservedRunningTime="2026-08-04 00:10:11.759542326 +0000 UTC m=+477.639989867" Aug 04 00:10:12.067785 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:12.067691 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:13.049019 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:13.048979 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:13.204321 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:13.204286 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:13.304463 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:13.304353 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:14.298994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:14.298958 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:17.601817 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:17.601763 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:17.702658 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:17.702620 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:21.006674 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:21.006631 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:21.064879 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:21.064841 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:21.595177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:21.595138 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:21.651086 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:21.651048 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:22.702129 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:22.702097 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:22.702535 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:22.702249 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-tvzb2" Aug 04 00:10:22.702535 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:22.702294 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:10:22.801078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:22.801049 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj"] Aug 04 00:10:22.801240 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:22.801213 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" podUID="f57c06c6-fc0b-4968-89b1-811cd60f2b9f" containerName="manager" containerID="cri-o://6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f" gracePeriod=10 Aug 04 00:10:23.049726 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.049701 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:23.134149 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.134115 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-extensions-socket-volume\") pod \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " Aug 04 00:10:23.134333 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.134239 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vxqr\" (UniqueName: \"kubernetes.io/projected/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-kube-api-access-6vxqr\") pod \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\" (UID: \"f57c06c6-fc0b-4968-89b1-811cd60f2b9f\") " Aug 04 00:10:23.134558 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.134528 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f57c06c6-fc0b-4968-89b1-811cd60f2b9f" (UID: "f57c06c6-fc0b-4968-89b1-811cd60f2b9f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:10:23.136205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.136181 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-kube-api-access-6vxqr" (OuterVolumeSpecName: "kube-api-access-6vxqr") pod "f57c06c6-fc0b-4968-89b1-811cd60f2b9f" (UID: "f57c06c6-fc0b-4968-89b1-811cd60f2b9f"). InnerVolumeSpecName "kube-api-access-6vxqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:10:23.235418 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.235340 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vxqr\" (UniqueName: \"kubernetes.io/projected/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-kube-api-access-6vxqr\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:10:23.235418 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.235368 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f57c06c6-fc0b-4968-89b1-811cd60f2b9f-extensions-socket-volume\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:10:23.760991 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.760953 2551 generic.go:356] "Generic (PLEG): container finished" podID="f57c06c6-fc0b-4968-89b1-811cd60f2b9f" containerID="6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f" exitCode=0 Aug 04 00:10:23.760991 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.760991 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" Aug 04 00:10:23.760991 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.760995 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" event={"ID":"f57c06c6-fc0b-4968-89b1-811cd60f2b9f","Type":"ContainerDied","Data":"6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f"} Aug 04 00:10:23.761456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.761046 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj" event={"ID":"f57c06c6-fc0b-4968-89b1-811cd60f2b9f","Type":"ContainerDied","Data":"93c5a001d67afd3cd2c107739868c3b2916c846a990d16f38a6761c01c8904a6"} Aug 04 00:10:23.761456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.761068 2551 scope.go:117] "RemoveContainer" containerID="6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f" Aug 04 00:10:23.769952 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.769937 2551 scope.go:117] "RemoveContainer" containerID="6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f" Aug 04 00:10:23.770200 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:10:23.770182 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f\": container with ID starting with 6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f not found: ID does not exist" containerID="6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f" Aug 04 00:10:23.770256 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.770208 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f"} err="failed to get container status \"6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f\": rpc error: code = NotFound desc = could not find container \"6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f\": container with ID starting with 6759dd34483ddbfeea1c1e50e05c511cdd6135d5e4a0eb5a9503233c67e8393f not found: ID does not exist" Aug 04 00:10:23.786693 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.786668 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj"] Aug 04 00:10:23.791345 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:23.791324 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-82jsj"] Aug 04 00:10:24.658991 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:24.658959 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f57c06c6-fc0b-4968-89b1-811cd60f2b9f" path="/var/lib/kubelet/pods/f57c06c6-fc0b-4968-89b1-811cd60f2b9f/volumes" Aug 04 00:10:25.901435 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:25.901404 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:25.997628 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:25.997595 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:26.100671 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:26.100634 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:32.397996 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:32.397964 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:32.497361 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:32.497330 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:37.132186 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:37.132152 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:37.235369 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:37.235339 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:39.118033 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.117994 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs"] Aug 04 00:10:39.118399 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.118385 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f57c06c6-fc0b-4968-89b1-811cd60f2b9f" containerName="manager" Aug 04 00:10:39.118399 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.118399 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f57c06c6-fc0b-4968-89b1-811cd60f2b9f" containerName="manager" Aug 04 00:10:39.118468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.118457 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f57c06c6-fc0b-4968-89b1-811cd60f2b9f" containerName="manager" Aug 04 00:10:39.121502 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.121483 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.124585 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.124563 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ctz5l\"" Aug 04 00:10:39.124735 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.124572 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 00:10:39.139937 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.139909 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs"] Aug 04 00:10:39.158202 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.158173 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:39.258608 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.258570 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:39.266621 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266596 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266717 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266628 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266717 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266685 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266717 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266708 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266860 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266759 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmqxf\" (UniqueName: \"kubernetes.io/projected/cb205534-bc74-4a36-b5cd-5ece6e5264ed-kube-api-access-tmqxf\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266860 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266817 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266964 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266859 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266964 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266884 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.266964 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.266912 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.367645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367613 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.367815 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367652 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.367815 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367705 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.367815 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367728 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.367815 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367749 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmqxf\" (UniqueName: \"kubernetes.io/projected/cb205534-bc74-4a36-b5cd-5ece6e5264ed-kube-api-access-tmqxf\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367915 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367959 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.367991 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.368023 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368353 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.368312 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368459 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.368417 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368575 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.368535 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.368600 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.368644 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.368630 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.370212 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.370183 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.370303 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.370248 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.375384 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.375364 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb205534-bc74-4a36-b5cd-5ece6e5264ed-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.375516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.375499 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmqxf\" (UniqueName: \"kubernetes.io/projected/cb205534-bc74-4a36-b5cd-5ece6e5264ed-kube-api-access-tmqxf\") pod \"maas-default-gateway-openshift-default-5ccb968c66-g68xs\" (UID: \"cb205534-bc74-4a36-b5cd-5ece6e5264ed\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.431337 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.431313 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:39.575701 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.575671 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs"] Aug 04 00:10:39.576603 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:39.576578 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb205534_bc74_4a36_b5cd_5ece6e5264ed.slice/crio-ca872fb7e4dfca8583df39c05bf476730c3bcb0c66bb435c34ddf4ef021a522e WatchSource:0}: Error finding container ca872fb7e4dfca8583df39c05bf476730c3bcb0c66bb435c34ddf4ef021a522e: Status 404 returned error can't find the container with id ca872fb7e4dfca8583df39c05bf476730c3bcb0c66bb435c34ddf4ef021a522e Aug 04 00:10:39.846585 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:39.846549 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" event={"ID":"cb205534-bc74-4a36-b5cd-5ece6e5264ed","Type":"ContainerStarted","Data":"ca872fb7e4dfca8583df39c05bf476730c3bcb0c66bb435c34ddf4ef021a522e"} Aug 04 00:10:40.104236 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:40.104160 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:40.162267 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:40.162229 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:40.939324 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:40.939289 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:41.008152 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:41.008115 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:45.544072 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.544035 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:45.609847 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.609814 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:45.619181 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.619144 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:10:45.619277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.619225 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:10:45.619277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.619261 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:10:45.879410 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.879375 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" event={"ID":"cb205534-bc74-4a36-b5cd-5ece6e5264ed","Type":"ContainerStarted","Data":"10360654d6f8cac6b1c5c010da4d10071799eddd84e5db295aa3d4308d60fb33"} Aug 04 00:10:45.902757 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.902699 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" podStartSLOduration=0.862378787 podStartE2EDuration="6.902683856s" totalImagesPullingTime="6.040305069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:45.901447377 +0000 UTC m=+511.781894916" watchObservedRunningTime="2026-08-04 00:10:45.902683856 +0000 UTC m=+511.783131395" Aug 04 00:10:45.959323 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:45.959287 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:46.021885 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.021853 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:46.351115 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.351018 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:46.422946 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.422906 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:46.431844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.431820 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:46.436389 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.436368 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:46.884784 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.884748 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:46.885779 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.885751 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:10:46.948980 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.948942 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:46.961457 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.961425 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-vdfd4"] Aug 04 00:10:46.965382 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.965361 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:46.967906 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.967885 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Aug 04 00:10:46.975446 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:46.975422 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-vdfd4"] Aug 04 00:10:47.027573 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.027540 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9813cba3-50bb-4f5a-bfcd-d966796d4b11-config-file\") pod \"limitador-limitador-859c4d96d6-vdfd4\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:47.027726 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.027608 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgx29\" (UniqueName: \"kubernetes.io/projected/9813cba3-50bb-4f5a-bfcd-d966796d4b11-kube-api-access-zgx29\") pod \"limitador-limitador-859c4d96d6-vdfd4\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:47.048902 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.048870 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:47.054553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.054528 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-vdfd4"] Aug 04 00:10:47.128515 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.128486 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgx29\" (UniqueName: \"kubernetes.io/projected/9813cba3-50bb-4f5a-bfcd-d966796d4b11-kube-api-access-zgx29\") pod \"limitador-limitador-859c4d96d6-vdfd4\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:47.128672 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.128541 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9813cba3-50bb-4f5a-bfcd-d966796d4b11-config-file\") pod \"limitador-limitador-859c4d96d6-vdfd4\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:47.129230 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.129194 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9813cba3-50bb-4f5a-bfcd-d966796d4b11-config-file\") pod \"limitador-limitador-859c4d96d6-vdfd4\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:47.140343 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.140279 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgx29\" (UniqueName: \"kubernetes.io/projected/9813cba3-50bb-4f5a-bfcd-d966796d4b11-kube-api-access-zgx29\") pod \"limitador-limitador-859c4d96d6-vdfd4\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:47.158397 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.158363 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:47.276276 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.276243 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:47.362375 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.362342 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:10:47.367825 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.367749 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.374915 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.374890 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:10:47.401639 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.401577 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:10:47.414999 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.414977 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-vdfd4"] Aug 04 00:10:47.416538 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:47.416513 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9813cba3_50bb_4f5a_bfcd_d966796d4b11.slice/crio-afff7f577f3552458404430843a210af3e4646d24d016e5f28eac182a915263a WatchSource:0}: Error finding container afff7f577f3552458404430843a210af3e4646d24d016e5f28eac182a915263a: Status 404 returned error can't find the container with id afff7f577f3552458404430843a210af3e4646d24d016e5f28eac182a915263a Aug 04 00:10:47.432073 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.432049 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/440cc743-e375-4c81-835e-b954959c91a6-config-file\") pod \"limitador-limitador-85d9db4d5d-qm2pq\" (UID: \"440cc743-e375-4c81-835e-b954959c91a6\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.432189 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.432107 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lss8\" (UniqueName: \"kubernetes.io/projected/440cc743-e375-4c81-835e-b954959c91a6-kube-api-access-4lss8\") pod \"limitador-limitador-85d9db4d5d-qm2pq\" (UID: \"440cc743-e375-4c81-835e-b954959c91a6\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.533345 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.533313 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lss8\" (UniqueName: \"kubernetes.io/projected/440cc743-e375-4c81-835e-b954959c91a6-kube-api-access-4lss8\") pod \"limitador-limitador-85d9db4d5d-qm2pq\" (UID: \"440cc743-e375-4c81-835e-b954959c91a6\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.533487 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.533367 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/440cc743-e375-4c81-835e-b954959c91a6-config-file\") pod \"limitador-limitador-85d9db4d5d-qm2pq\" (UID: \"440cc743-e375-4c81-835e-b954959c91a6\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.533945 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.533925 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/440cc743-e375-4c81-835e-b954959c91a6-config-file\") pod \"limitador-limitador-85d9db4d5d-qm2pq\" (UID: \"440cc743-e375-4c81-835e-b954959c91a6\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.541421 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.541397 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lss8\" (UniqueName: \"kubernetes.io/projected/440cc743-e375-4c81-835e-b954959c91a6-kube-api-access-4lss8\") pod \"limitador-limitador-85d9db4d5d-qm2pq\" (UID: \"440cc743-e375-4c81-835e-b954959c91a6\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.685248 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.685177 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:47.784144 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.784113 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:47.817567 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.817543 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:10:47.819049 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:47.819018 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440cc743_e375_4c81_835e_b954959c91a6.slice/crio-d0e7ca30ee9e7634f2da30ce236faac2357f907fe8cdcd0c82f5b3fe3f23809d WatchSource:0}: Error finding container d0e7ca30ee9e7634f2da30ce236faac2357f907fe8cdcd0c82f5b3fe3f23809d: Status 404 returned error can't find the container with id d0e7ca30ee9e7634f2da30ce236faac2357f907fe8cdcd0c82f5b3fe3f23809d Aug 04 00:10:47.885445 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.885412 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:47.890933 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.890903 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" event={"ID":"440cc743-e375-4c81-835e-b954959c91a6","Type":"ContainerStarted","Data":"d0e7ca30ee9e7634f2da30ce236faac2357f907fe8cdcd0c82f5b3fe3f23809d"} Aug 04 00:10:47.892113 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.892089 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" event={"ID":"9813cba3-50bb-4f5a-bfcd-d966796d4b11","Type":"ContainerStarted","Data":"afff7f577f3552458404430843a210af3e4646d24d016e5f28eac182a915263a"} Aug 04 00:10:47.908594 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.908564 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8hf4g"] Aug 04 00:10:47.913576 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.913560 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" Aug 04 00:10:47.916216 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.916197 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7jlmt\"" Aug 04 00:10:47.918393 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.918370 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8hf4g"] Aug 04 00:10:47.937247 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:47.937189 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn82z\" (UniqueName: \"kubernetes.io/projected/f3712e66-2059-4595-9319-f29840419fc3-kube-api-access-cn82z\") pod \"authorino-c548b5d7d-8hf4g\" (UID: \"f3712e66-2059-4595-9319-f29840419fc3\") " pod="kuadrant-system/authorino-c548b5d7d-8hf4g" Aug 04 00:10:48.037893 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:48.037858 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn82z\" (UniqueName: \"kubernetes.io/projected/f3712e66-2059-4595-9319-f29840419fc3-kube-api-access-cn82z\") pod \"authorino-c548b5d7d-8hf4g\" (UID: \"f3712e66-2059-4595-9319-f29840419fc3\") " pod="kuadrant-system/authorino-c548b5d7d-8hf4g" Aug 04 00:10:48.046224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:48.046204 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn82z\" (UniqueName: \"kubernetes.io/projected/f3712e66-2059-4595-9319-f29840419fc3-kube-api-access-cn82z\") pod \"authorino-c548b5d7d-8hf4g\" (UID: \"f3712e66-2059-4595-9319-f29840419fc3\") " pod="kuadrant-system/authorino-c548b5d7d-8hf4g" Aug 04 00:10:48.224213 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:48.224136 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" Aug 04 00:10:48.351931 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:48.351905 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8hf4g"] Aug 04 00:10:48.353377 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:10:48.353347 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3712e66_2059_4595_9319_f29840419fc3.slice/crio-012ebd148d4cb7af97830e8bc7a641278fc67eddb2c3a356d3ff81fcd7a59f61 WatchSource:0}: Error finding container 012ebd148d4cb7af97830e8bc7a641278fc67eddb2c3a356d3ff81fcd7a59f61: Status 404 returned error can't find the container with id 012ebd148d4cb7af97830e8bc7a641278fc67eddb2c3a356d3ff81fcd7a59f61 Aug 04 00:10:48.898298 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:48.898268 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" event={"ID":"f3712e66-2059-4595-9319-f29840419fc3","Type":"ContainerStarted","Data":"012ebd148d4cb7af97830e8bc7a641278fc67eddb2c3a356d3ff81fcd7a59f61"} Aug 04 00:10:49.906210 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:49.906154 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" event={"ID":"440cc743-e375-4c81-835e-b954959c91a6","Type":"ContainerStarted","Data":"1fa48a8e8358ea53c4680c947d8e92e1fddf0debf7310cb28b1d20a0ad0d1e96"} Aug 04 00:10:49.906675 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:49.906355 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:10:49.908011 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:49.907981 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" event={"ID":"9813cba3-50bb-4f5a-bfcd-d966796d4b11","Type":"ContainerStarted","Data":"51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696"} Aug 04 00:10:49.908379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:49.908069 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:10:49.928554 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:49.928497 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" podStartSLOduration=1.846939619 podStartE2EDuration="2.928483677s" totalImagesPullingTime="1.081544058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:49.926620337 +0000 UTC m=+515.807067878" watchObservedRunningTime="2026-08-04 00:10:49.928483677 +0000 UTC m=+515.808931218" Aug 04 00:10:49.954844 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:49.954774 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" podStartSLOduration=2.472310702 podStartE2EDuration="3.954754156s" totalImagesPullingTime="1.482443454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:49.952908077 +0000 UTC m=+515.833355639" watchObservedRunningTime="2026-08-04 00:10:49.954754156 +0000 UTC m=+515.835201696" Aug 04 00:10:51.818996 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:51.818876 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:51.875987 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:51.875951 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:51.925910 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:51.925872 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" event={"ID":"f3712e66-2059-4595-9319-f29840419fc3","Type":"ContainerStarted","Data":"c4daa772982bdc4d571c01ac9125d36bc34061766175a656c8d0af7d6ca8b358"} Aug 04 00:10:51.941309 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:51.941266 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" podStartSLOduration=1.8972588030000002 podStartE2EDuration="4.941251098s" totalImagesPullingTime="3.043992295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:10:51.939698533 +0000 UTC m=+517.820146065" watchObservedRunningTime="2026-08-04 00:10:51.941251098 +0000 UTC m=+517.821698665" Aug 04 00:10:54.076584 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:54.076537 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:56.574054 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:56.574001 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:10:56.673846 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:10:56.673812 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:00.916118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:00.916080 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:11:00.916480 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:00.916257 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq" Aug 04 00:11:00.975112 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:00.975080 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-vdfd4"] Aug 04 00:11:00.978255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:00.978202 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" podUID="9813cba3-50bb-4f5a-bfcd-d966796d4b11" containerName="limitador" containerID="cri-o://51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696" gracePeriod=30 Aug 04 00:11:01.920443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.920419 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:11:01.957232 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.957208 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgx29\" (UniqueName: \"kubernetes.io/projected/9813cba3-50bb-4f5a-bfcd-d966796d4b11-kube-api-access-zgx29\") pod \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " Aug 04 00:11:01.957359 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.957256 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9813cba3-50bb-4f5a-bfcd-d966796d4b11-config-file\") pod \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\" (UID: \"9813cba3-50bb-4f5a-bfcd-d966796d4b11\") " Aug 04 00:11:01.957633 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.957611 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9813cba3-50bb-4f5a-bfcd-d966796d4b11-config-file" (OuterVolumeSpecName: "config-file") pod "9813cba3-50bb-4f5a-bfcd-d966796d4b11" (UID: "9813cba3-50bb-4f5a-bfcd-d966796d4b11"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:11:01.959206 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.959179 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9813cba3-50bb-4f5a-bfcd-d966796d4b11-kube-api-access-zgx29" (OuterVolumeSpecName: "kube-api-access-zgx29") pod "9813cba3-50bb-4f5a-bfcd-d966796d4b11" (UID: "9813cba3-50bb-4f5a-bfcd-d966796d4b11"). InnerVolumeSpecName "kube-api-access-zgx29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:11:01.984172 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.984148 2551 generic.go:356] "Generic (PLEG): container finished" podID="9813cba3-50bb-4f5a-bfcd-d966796d4b11" containerID="51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696" exitCode=0 Aug 04 00:11:01.984268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.984191 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" Aug 04 00:11:01.984268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.984217 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" event={"ID":"9813cba3-50bb-4f5a-bfcd-d966796d4b11","Type":"ContainerDied","Data":"51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696"} Aug 04 00:11:01.984268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.984257 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-vdfd4" event={"ID":"9813cba3-50bb-4f5a-bfcd-d966796d4b11","Type":"ContainerDied","Data":"afff7f577f3552458404430843a210af3e4646d24d016e5f28eac182a915263a"} Aug 04 00:11:01.984381 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.984280 2551 scope.go:117] "RemoveContainer" containerID="51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696" Aug 04 00:11:01.995677 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.994127 2551 scope.go:117] "RemoveContainer" containerID="51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696" Aug 04 00:11:01.995677 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:11:01.995107 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696\": container with ID starting with 51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696 not found: ID does not exist" containerID="51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696" Aug 04 00:11:01.995677 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:01.995142 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696"} err="failed to get container status \"51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696\": rpc error: code = NotFound desc = could not find container \"51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696\": container with ID starting with 51d52815ad0287d42a56206b757f2228166353faabe8ef47626c615d26a6e696 not found: ID does not exist" Aug 04 00:11:02.011762 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:02.011740 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-vdfd4"] Aug 04 00:11:02.015691 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:02.015672 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-vdfd4"] Aug 04 00:11:02.058111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:02.058050 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgx29\" (UniqueName: \"kubernetes.io/projected/9813cba3-50bb-4f5a-bfcd-d966796d4b11-kube-api-access-zgx29\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:11:02.058111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:02.058074 2551 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9813cba3-50bb-4f5a-bfcd-d966796d4b11-config-file\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:11:02.658895 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:02.658864 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9813cba3-50bb-4f5a-bfcd-d966796d4b11" path="/var/lib/kubelet/pods/9813cba3-50bb-4f5a-bfcd-d966796d4b11/volumes" Aug 04 00:11:03.098960 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:03.098873 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:03.199978 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:03.199933 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:03.586065 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:03.586029 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:16.126977 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:16.126928 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:16.335222 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:16.335166 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:17.864566 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:17.864525 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8hf4g"] Aug 04 00:11:17.865023 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:17.864749 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" podUID="f3712e66-2059-4595-9319-f29840419fc3" containerName="authorino" containerID="cri-o://c4daa772982bdc4d571c01ac9125d36bc34061766175a656c8d0af7d6ca8b358" gracePeriod=30 Aug 04 00:11:18.077149 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:18.077118 2551 generic.go:356] "Generic (PLEG): container finished" podID="f3712e66-2059-4595-9319-f29840419fc3" containerID="c4daa772982bdc4d571c01ac9125d36bc34061766175a656c8d0af7d6ca8b358" exitCode=0 Aug 04 00:11:18.077291 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:18.077187 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" event={"ID":"f3712e66-2059-4595-9319-f29840419fc3","Type":"ContainerDied","Data":"c4daa772982bdc4d571c01ac9125d36bc34061766175a656c8d0af7d6ca8b358"} Aug 04 00:11:18.112662 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:18.112641 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" Aug 04 00:11:18.193177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:18.193149 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn82z\" (UniqueName: \"kubernetes.io/projected/f3712e66-2059-4595-9319-f29840419fc3-kube-api-access-cn82z\") pod \"f3712e66-2059-4595-9319-f29840419fc3\" (UID: \"f3712e66-2059-4595-9319-f29840419fc3\") " Aug 04 00:11:18.195145 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:18.195116 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3712e66-2059-4595-9319-f29840419fc3-kube-api-access-cn82z" (OuterVolumeSpecName: "kube-api-access-cn82z") pod "f3712e66-2059-4595-9319-f29840419fc3" (UID: "f3712e66-2059-4595-9319-f29840419fc3"). InnerVolumeSpecName "kube-api-access-cn82z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:11:18.294182 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:18.294146 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn82z\" (UniqueName: \"kubernetes.io/projected/f3712e66-2059-4595-9319-f29840419fc3-kube-api-access-cn82z\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:11:19.083040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:19.082951 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" Aug 04 00:11:19.083040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:19.082995 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8hf4g" event={"ID":"f3712e66-2059-4595-9319-f29840419fc3","Type":"ContainerDied","Data":"012ebd148d4cb7af97830e8bc7a641278fc67eddb2c3a356d3ff81fcd7a59f61"} Aug 04 00:11:19.083517 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:19.083058 2551 scope.go:117] "RemoveContainer" containerID="c4daa772982bdc4d571c01ac9125d36bc34061766175a656c8d0af7d6ca8b358" Aug 04 00:11:19.101171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:19.101148 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8hf4g"] Aug 04 00:11:19.104957 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:19.104935 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8hf4g"] Aug 04 00:11:20.659127 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:20.659096 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3712e66-2059-4595-9319-f29840419fc3" path="/var/lib/kubelet/pods/f3712e66-2059-4595-9319-f29840419fc3/volumes" Aug 04 00:11:21.011672 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:21.011588 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:21.112742 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:21.112707 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:22.480022 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.479984 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:22.494976 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.494949 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-794485cd76-slx55"] Aug 04 00:11:22.495298 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.495286 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9813cba3-50bb-4f5a-bfcd-d966796d4b11" containerName="limitador" Aug 04 00:11:22.495298 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.495299 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9813cba3-50bb-4f5a-bfcd-d966796d4b11" containerName="limitador" Aug 04 00:11:22.495436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.495307 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3712e66-2059-4595-9319-f29840419fc3" containerName="authorino" Aug 04 00:11:22.495436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.495312 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3712e66-2059-4595-9319-f29840419fc3" containerName="authorino" Aug 04 00:11:22.495436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.495363 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9813cba3-50bb-4f5a-bfcd-d966796d4b11" containerName="limitador" Aug 04 00:11:22.495436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.495370 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3712e66-2059-4595-9319-f29840419fc3" containerName="authorino" Aug 04 00:11:22.501434 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.501414 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.504200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.504180 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 00:11:22.504299 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.504241 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 00:11:22.504364 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.504297 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d6x2w\"" Aug 04 00:11:22.513127 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.513102 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794485cd76-slx55"] Aug 04 00:11:22.546368 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.546320 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:22.597253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.597229 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:22.631333 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.631305 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg5q7\" (UniqueName: \"kubernetes.io/projected/d73b4de8-639a-4496-bf94-1808bf38574d-kube-api-access-jg5q7\") pod \"maas-controller-794485cd76-slx55\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.631456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.631353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d73b4de8-639a-4496-bf94-1808bf38574d-webhook-cert\") pod \"maas-controller-794485cd76-slx55\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.660319 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.660295 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-89565659c-wh97j"] Aug 04 00:11:22.664587 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.664566 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:22.681972 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.681944 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-89565659c-wh97j"] Aug 04 00:11:22.722804 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.722740 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:22.731842 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.731773 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg5q7\" (UniqueName: \"kubernetes.io/projected/d73b4de8-639a-4496-bf94-1808bf38574d-kube-api-access-jg5q7\") pod \"maas-controller-794485cd76-slx55\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.731842 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.731835 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d73b4de8-639a-4496-bf94-1808bf38574d-webhook-cert\") pod \"maas-controller-794485cd76-slx55\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.734123 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.734106 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d73b4de8-639a-4496-bf94-1808bf38574d-webhook-cert\") pod \"maas-controller-794485cd76-slx55\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.740060 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.740040 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg5q7\" (UniqueName: \"kubernetes.io/projected/d73b4de8-639a-4496-bf94-1808bf38574d-kube-api-access-jg5q7\") pod \"maas-controller-794485cd76-slx55\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.812983 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.812955 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:22.833230 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.833203 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe048780-e1ae-4b93-bcff-6757f660307e-webhook-cert\") pod \"maas-controller-89565659c-wh97j\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:22.833337 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.833303 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftwk2\" (UniqueName: \"kubernetes.io/projected/fe048780-e1ae-4b93-bcff-6757f660307e-kube-api-access-ftwk2\") pod \"maas-controller-89565659c-wh97j\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:22.934266 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.934234 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftwk2\" (UniqueName: \"kubernetes.io/projected/fe048780-e1ae-4b93-bcff-6757f660307e-kube-api-access-ftwk2\") pod \"maas-controller-89565659c-wh97j\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:22.934414 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.934303 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe048780-e1ae-4b93-bcff-6757f660307e-webhook-cert\") pod \"maas-controller-89565659c-wh97j\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:22.936587 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.936562 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe048780-e1ae-4b93-bcff-6757f660307e-webhook-cert\") pod \"maas-controller-89565659c-wh97j\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:22.942255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.942232 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794485cd76-slx55"] Aug 04 00:11:22.944180 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:11:22.944152 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73b4de8_639a_4496_bf94_1808bf38574d.slice/crio-b76b825b46a6586aa51676a8000a38a565b4beeffb4a86452724c74e59295b07 WatchSource:0}: Error finding container b76b825b46a6586aa51676a8000a38a565b4beeffb4a86452724c74e59295b07: Status 404 returned error can't find the container with id b76b825b46a6586aa51676a8000a38a565b4beeffb4a86452724c74e59295b07 Aug 04 00:11:22.944301 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.944282 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftwk2\" (UniqueName: \"kubernetes.io/projected/fe048780-e1ae-4b93-bcff-6757f660307e-kube-api-access-ftwk2\") pod \"maas-controller-89565659c-wh97j\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:22.974609 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:22.974587 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:23.104695 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:23.104665 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-89565659c-wh97j"] Aug 04 00:11:23.104958 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:11:23.104934 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe048780_e1ae_4b93_bcff_6757f660307e.slice/crio-7ef6153d006def962fe4ebba40d24e293ba57acc4e17dae606448486fbc8b6de WatchSource:0}: Error finding container 7ef6153d006def962fe4ebba40d24e293ba57acc4e17dae606448486fbc8b6de: Status 404 returned error can't find the container with id 7ef6153d006def962fe4ebba40d24e293ba57acc4e17dae606448486fbc8b6de Aug 04 00:11:23.106522 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:23.106492 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794485cd76-slx55" event={"ID":"d73b4de8-639a-4496-bf94-1808bf38574d","Type":"ContainerStarted","Data":"b76b825b46a6586aa51676a8000a38a565b4beeffb4a86452724c74e59295b07"} Aug 04 00:11:24.113983 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:24.113944 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89565659c-wh97j" event={"ID":"fe048780-e1ae-4b93-bcff-6757f660307e","Type":"ContainerStarted","Data":"7ef6153d006def962fe4ebba40d24e293ba57acc4e17dae606448486fbc8b6de"} Aug 04 00:11:24.636465 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:24.636425 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:24.745157 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:24.745074 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:25.121493 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:25.121396 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89565659c-wh97j" event={"ID":"fe048780-e1ae-4b93-bcff-6757f660307e","Type":"ContainerStarted","Data":"61252794aa7b17275b2991796bf049fca98e588bdb365829cd88023196556754"} Aug 04 00:11:25.121493 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:25.121454 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:25.122852 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:25.122827 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794485cd76-slx55" event={"ID":"d73b4de8-639a-4496-bf94-1808bf38574d","Type":"ContainerStarted","Data":"08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69"} Aug 04 00:11:25.122920 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:25.122876 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:25.140416 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:25.140375 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-89565659c-wh97j" podStartSLOduration=1.79587983 podStartE2EDuration="3.140361962s" totalImagesPullingTime="1.344482132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:11:25.138304872 +0000 UTC m=+551.018752413" watchObservedRunningTime="2026-08-04 00:11:25.140361962 +0000 UTC m=+551.020809501" Aug 04 00:11:25.157502 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:25.157461 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-794485cd76-slx55" podStartSLOduration=1.649139338 podStartE2EDuration="3.157449038s" totalImagesPullingTime="1.5083097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:11:25.157024737 +0000 UTC m=+551.037472268" watchObservedRunningTime="2026-08-04 00:11:25.157449038 +0000 UTC m=+551.037896577" Aug 04 00:11:32.506313 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:32.506272 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:36.130954 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.130921 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:11:36.131420 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.130989 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:36.224611 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.224578 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794485cd76-slx55"] Aug 04 00:11:36.224803 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.224774 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-794485cd76-slx55" podUID="d73b4de8-639a-4496-bf94-1808bf38574d" containerName="manager" containerID="cri-o://08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69" gracePeriod=30 Aug 04 00:11:36.465284 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.465260 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:36.553789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.553750 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d73b4de8-639a-4496-bf94-1808bf38574d-webhook-cert\") pod \"d73b4de8-639a-4496-bf94-1808bf38574d\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " Aug 04 00:11:36.553975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.553826 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg5q7\" (UniqueName: \"kubernetes.io/projected/d73b4de8-639a-4496-bf94-1808bf38574d-kube-api-access-jg5q7\") pod \"d73b4de8-639a-4496-bf94-1808bf38574d\" (UID: \"d73b4de8-639a-4496-bf94-1808bf38574d\") " Aug 04 00:11:36.556009 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.555982 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d73b4de8-639a-4496-bf94-1808bf38574d-kube-api-access-jg5q7" (OuterVolumeSpecName: "kube-api-access-jg5q7") pod "d73b4de8-639a-4496-bf94-1808bf38574d" (UID: "d73b4de8-639a-4496-bf94-1808bf38574d"). InnerVolumeSpecName "kube-api-access-jg5q7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:11:36.556080 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.556044 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d73b4de8-639a-4496-bf94-1808bf38574d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d73b4de8-639a-4496-bf94-1808bf38574d" (UID: "d73b4de8-639a-4496-bf94-1808bf38574d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:11:36.654397 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.654260 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg5q7\" (UniqueName: \"kubernetes.io/projected/d73b4de8-639a-4496-bf94-1808bf38574d-kube-api-access-jg5q7\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:11:36.654397 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:36.654291 2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d73b4de8-639a-4496-bf94-1808bf38574d-webhook-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:11:37.189036 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.188999 2551 generic.go:356] "Generic (PLEG): container finished" podID="d73b4de8-639a-4496-bf94-1808bf38574d" containerID="08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69" exitCode=0 Aug 04 00:11:37.189390 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.189047 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794485cd76-slx55" Aug 04 00:11:37.189390 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.189047 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794485cd76-slx55" event={"ID":"d73b4de8-639a-4496-bf94-1808bf38574d","Type":"ContainerDied","Data":"08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69"} Aug 04 00:11:37.189390 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.189170 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794485cd76-slx55" event={"ID":"d73b4de8-639a-4496-bf94-1808bf38574d","Type":"ContainerDied","Data":"b76b825b46a6586aa51676a8000a38a565b4beeffb4a86452724c74e59295b07"} Aug 04 00:11:37.189390 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.189197 2551 scope.go:117] "RemoveContainer" containerID="08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69" Aug 04 00:11:37.197947 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.197928 2551 scope.go:117] "RemoveContainer" containerID="08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69" Aug 04 00:11:37.198213 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:11:37.198196 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69\": container with ID starting with 08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69 not found: ID does not exist" containerID="08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69" Aug 04 00:11:37.198252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.198225 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69"} err="failed to get container status \"08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69\": rpc error: code = NotFound desc = could not find container \"08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69\": container with ID starting with 08eb50f474de31bf7c7e711d3004cb7db7ac42004ef2ed99558d6816f6912d69 not found: ID does not exist" Aug 04 00:11:37.206232 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.206210 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794485cd76-slx55"] Aug 04 00:11:37.210879 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:37.210859 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-794485cd76-slx55"] Aug 04 00:11:38.659060 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:38.659030 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d73b4de8-639a-4496-bf94-1808bf38574d" path="/var/lib/kubelet/pods/d73b4de8-639a-4496-bf94-1808bf38574d/volumes" Aug 04 00:11:43.436726 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:43.436665 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:57.257489 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:57.257452 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:57.358301 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:57.358255 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:58.757387 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:58.757340 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:58.859248 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:58.859214 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:59.057729 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.057654 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6"] Aug 04 00:11:59.058022 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.058009 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73b4de8-639a-4496-bf94-1808bf38574d" containerName="manager" Aug 04 00:11:59.058069 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.058023 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73b4de8-639a-4496-bf94-1808bf38574d" containerName="manager" Aug 04 00:11:59.058104 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.058090 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73b4de8-639a-4496-bf94-1808bf38574d" containerName="manager" Aug 04 00:11:59.063963 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.063945 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.066334 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.066313 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 00:11:59.067510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.067487 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Aug 04 00:11:59.067601 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.067526 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 00:11:59.067601 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.067537 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kqm87\"" Aug 04 00:11:59.072010 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.071987 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6"] Aug 04 00:11:59.125053 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.125027 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f795a71-27c6-40de-b980-e6cbfb421008-maas-api-tls\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.125202 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.125060 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f795a71-27c6-40de-b980-e6cbfb421008-openshift-service-ca\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.125202 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.125092 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwt72\" (UniqueName: \"kubernetes.io/projected/8f795a71-27c6-40de-b980-e6cbfb421008-kube-api-access-xwt72\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.226144 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.226112 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f795a71-27c6-40de-b980-e6cbfb421008-maas-api-tls\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.226317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.226148 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f795a71-27c6-40de-b980-e6cbfb421008-openshift-service-ca\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.226317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.226183 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwt72\" (UniqueName: \"kubernetes.io/projected/8f795a71-27c6-40de-b980-e6cbfb421008-kube-api-access-xwt72\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.226909 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.226884 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f795a71-27c6-40de-b980-e6cbfb421008-openshift-service-ca\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.228591 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.228566 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f795a71-27c6-40de-b980-e6cbfb421008-maas-api-tls\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.234557 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.234533 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwt72\" (UniqueName: \"kubernetes.io/projected/8f795a71-27c6-40de-b980-e6cbfb421008-kube-api-access-xwt72\") pod \"maas-api-5dc96dd749-lwtd6\" (UID: \"8f795a71-27c6-40de-b980-e6cbfb421008\") " pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.354818 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.354709 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:59.375277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.375242 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:11:59.449016 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.448984 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:11:59.527863 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.518590 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6"] Aug 04 00:11:59.609871 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:11:59.609772 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:12:00.312831 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:00.312744 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" event={"ID":"8f795a71-27c6-40de-b980-e6cbfb421008","Type":"ContainerStarted","Data":"cf27962231ed2640e2f693bec6ca14e1ef0f3a7a78b1489adaf29bafb79c9c6a"} Aug 04 00:12:03.333830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:03.333774 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" event={"ID":"8f795a71-27c6-40de-b980-e6cbfb421008","Type":"ContainerStarted","Data":"f47be8ddb69d94bed23e43e1bd803af44c0c0e564b479b7cf0c8231beff02c3f"} Aug 04 00:12:03.334205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:03.333867 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:12:03.353488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:03.353438 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" podStartSLOduration=1.457775136 podStartE2EDuration="4.353421275s" totalImagesPullingTime="2.895646139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:12:03.351312086 +0000 UTC m=+589.231759640" watchObservedRunningTime="2026-08-04 00:12:03.353421275 +0000 UTC m=+589.233868815" Aug 04 00:12:09.344261 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:09.344231 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5dc96dd749-lwtd6" Aug 04 00:12:09.364611 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:09.364582 2551 generic.go:356] "Generic (PLEG): container finished" podID="cb205534-bc74-4a36-b5cd-5ece6e5264ed" containerID="10360654d6f8cac6b1c5c010da4d10071799eddd84e5db295aa3d4308d60fb33" exitCode=0 Aug 04 00:12:09.364777 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:09.364616 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" event={"ID":"cb205534-bc74-4a36-b5cd-5ece6e5264ed","Type":"ContainerDied","Data":"10360654d6f8cac6b1c5c010da4d10071799eddd84e5db295aa3d4308d60fb33"} Aug 04 00:12:09.365014 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:09.364995 2551 scope.go:117] "RemoveContainer" containerID="10360654d6f8cac6b1c5c010da4d10071799eddd84e5db295aa3d4308d60fb33" Aug 04 00:12:09.365961 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:09.365933 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:12:09.366028 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:09.365998 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:12:09.366065 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:09.366031 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:12:10.284305 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:10.284116 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:10.375956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:10.375007 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" event={"ID":"cb205534-bc74-4a36-b5cd-5ece6e5264ed","Type":"ContainerStarted","Data":"74e54e1b9251d96c6381aa5b4c7646fef8b19e41beafe455cf71f4b26c408a4b"} Aug 04 00:12:10.387269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:10.387178 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:10.431517 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:10.431484 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:12:10.431763 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:10.431738 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-g68xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Aug 04 00:12:10.431855 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:10.431807 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" podUID="cb205534-bc74-4a36-b5cd-5ece6e5264ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Aug 04 00:12:11.431790 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:11.431750 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-g68xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Aug 04 00:12:11.432200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:11.431841 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" podUID="cb205534-bc74-4a36-b5cd-5ece6e5264ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Aug 04 00:12:12.432213 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:12.432178 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-g68xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Aug 04 00:12:12.432567 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:12.432241 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" podUID="cb205534-bc74-4a36-b5cd-5ece6e5264ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Aug 04 00:12:13.432001 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:13.431964 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-g68xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Aug 04 00:12:13.432174 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:13.432032 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" podUID="cb205534-bc74-4a36-b5cd-5ece6e5264ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Aug 04 00:12:14.432432 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:14.432398 2551 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-g68xs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Aug 04 00:12:14.432830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:14.432455 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" podUID="cb205534-bc74-4a36-b5cd-5ece6e5264ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Aug 04 00:12:15.457263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.457224 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:12:15.457667 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.457635 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:12:15.458417 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.458397 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-g68xs" Aug 04 00:12:15.588078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.588051 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2"] Aug 04 00:12:15.594502 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.594486 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.597643 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.597615 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Aug 04 00:12:15.597779 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.597653 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 00:12:15.597990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.597969 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nc2tq\"" Aug 04 00:12:15.598146 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.598124 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 00:12:15.600128 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.600106 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2"] Aug 04 00:12:15.611789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.611763 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:15.699178 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.699143 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.699355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.699188 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.699355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.699218 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.699355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.699316 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.699527 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.699361 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7da78db6-0f82-4b0e-a846-025a6fb10f8f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.699527 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.699431 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8zld\" (UniqueName: \"kubernetes.io/projected/7da78db6-0f82-4b0e-a846-025a6fb10f8f-kube-api-access-v8zld\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800259 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800187 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7da78db6-0f82-4b0e-a846-025a6fb10f8f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800259 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800233 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8zld\" (UniqueName: \"kubernetes.io/projected/7da78db6-0f82-4b0e-a846-025a6fb10f8f-kube-api-access-v8zld\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800470 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800276 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800470 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800302 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800470 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800323 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800470 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800371 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800737 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800694 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800849 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800745 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.800849 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.800788 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.802536 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.802514 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7da78db6-0f82-4b0e-a846-025a6fb10f8f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.802789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.802772 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7da78db6-0f82-4b0e-a846-025a6fb10f8f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.808359 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.808335 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8zld\" (UniqueName: \"kubernetes.io/projected/7da78db6-0f82-4b0e-a846-025a6fb10f8f-kube-api-access-v8zld\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2\" (UID: \"7da78db6-0f82-4b0e-a846-025a6fb10f8f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:15.907031 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:15.906992 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:16.042178 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:16.042147 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2"] Aug 04 00:12:16.043462 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:12:16.043439 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da78db6_0f82_4b0e_a846_025a6fb10f8f.slice/crio-bb44f9a43f0fb150e9fd416f1c089b03c3b669c2ce20a36d9d12f2d21f6b53a0 WatchSource:0}: Error finding container bb44f9a43f0fb150e9fd416f1c089b03c3b669c2ce20a36d9d12f2d21f6b53a0: Status 404 returned error can't find the container with id bb44f9a43f0fb150e9fd416f1c089b03c3b669c2ce20a36d9d12f2d21f6b53a0 Aug 04 00:12:16.133153 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:16.133124 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:12:16.416553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:16.416467 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" event={"ID":"7da78db6-0f82-4b0e-a846-025a6fb10f8f","Type":"ContainerStarted","Data":"bb44f9a43f0fb150e9fd416f1c089b03c3b669c2ce20a36d9d12f2d21f6b53a0"} Aug 04 00:12:17.424772 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:17.424687 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" event={"ID":"7da78db6-0f82-4b0e-a846-025a6fb10f8f","Type":"ContainerStarted","Data":"2dd9204b15174b16033eba383bc9bb947b377614eb7d81f50ed98c97c4d38dae"} Aug 04 00:12:17.442531 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:17.442482 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" podStartSLOduration=1.314911431 podStartE2EDuration="2.442466602s" totalImagesPullingTime="1.127555171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:12:17.441139624 +0000 UTC m=+603.321587164" watchObservedRunningTime="2026-08-04 00:12:17.442466602 +0000 UTC m=+603.322914141" Aug 04 00:12:24.834647 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:24.834588 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:25.620226 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:25.620189 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:12:25.907328 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:25.907288 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:25.907848 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:25.907357 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:25.919960 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:25.919936 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:26.484039 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:26.484006 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2" Aug 04 00:12:37.795299 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.795263 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d"] Aug 04 00:12:37.799033 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.799015 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:37.801850 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.801826 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Aug 04 00:12:37.809667 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.809644 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d"] Aug 04 00:12:37.831087 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.831059 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:37.909905 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.909873 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:37.910042 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.909917 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:37.910042 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.909975 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:37.910164 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.910067 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:37.910164 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.910100 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86db44ce-bfd9-4848-8c56-0412cbff28d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:37.910164 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:37.910147 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtbmc\" (UniqueName: \"kubernetes.io/projected/86db44ce-bfd9-4848-8c56-0412cbff28d9-kube-api-access-qtbmc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.011541 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.011509 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.011541 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.011541 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86db44ce-bfd9-4848-8c56-0412cbff28d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.011720 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.011558 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtbmc\" (UniqueName: \"kubernetes.io/projected/86db44ce-bfd9-4848-8c56-0412cbff28d9-kube-api-access-qtbmc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.011816 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.011769 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.011870 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.011852 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.011908 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.011897 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.012002 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.011982 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.012109 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.012090 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.012222 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.012202 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.013995 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.013969 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86db44ce-bfd9-4848-8c56-0412cbff28d9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.014096 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.014072 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86db44ce-bfd9-4848-8c56-0412cbff28d9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.019960 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.019941 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtbmc\" (UniqueName: \"kubernetes.io/projected/86db44ce-bfd9-4848-8c56-0412cbff28d9-kube-api-access-qtbmc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pn25d\" (UID: \"86db44ce-bfd9-4848-8c56-0412cbff28d9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.110570 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.110484 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:38.274149 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:12:38.274116 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86db44ce_bfd9_4848_8c56_0412cbff28d9.slice/crio-595ca0cb0f2bfa763be91099ded14a346e2e5988fb4f1915b1619828a464b4fa WatchSource:0}: Error finding container 595ca0cb0f2bfa763be91099ded14a346e2e5988fb4f1915b1619828a464b4fa: Status 404 returned error can't find the container with id 595ca0cb0f2bfa763be91099ded14a346e2e5988fb4f1915b1619828a464b4fa Aug 04 00:12:38.274279 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.274222 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d"] Aug 04 00:12:38.275575 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.275554 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 00:12:38.334136 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.334104 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:12:38.542781 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.542738 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" event={"ID":"86db44ce-bfd9-4848-8c56-0412cbff28d9","Type":"ContainerStarted","Data":"6069926043a53132b964f9dc08e7c3f867c872265c733b7193f2c2e72c1ff4b0"} Aug 04 00:12:38.542953 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.542816 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" event={"ID":"86db44ce-bfd9-4848-8c56-0412cbff28d9","Type":"ContainerStarted","Data":"595ca0cb0f2bfa763be91099ded14a346e2e5988fb4f1915b1619828a464b4fa"} Aug 04 00:12:38.562307 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:38.562265 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" podStartSLOduration=1.39339923 podStartE2EDuration="1.562250288s" totalImagesPullingTime="168.851058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:12:38.561629131 +0000 UTC m=+624.442076709" watchObservedRunningTime="2026-08-04 00:12:38.562250288 +0000 UTC m=+624.442697828" Aug 04 00:12:48.111486 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:48.111447 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:48.111968 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:48.111507 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:48.123909 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:48.123887 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:48.607145 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:48.607112 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pn25d" Aug 04 00:12:50.006386 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:50.006347 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:51.129909 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:51.129872 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:12:54.102199 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.102166 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7"] Aug 04 00:12:54.106019 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.105999 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.108711 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.108686 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Aug 04 00:12:54.116850 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.116826 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7"] Aug 04 00:12:54.129367 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.129340 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:54.256616 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.256580 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.256832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.256624 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00355fc0-7beb-4357-8aec-62cec0b609ad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.256832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.256655 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.256832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.256681 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.256832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.256733 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn75p\" (UniqueName: \"kubernetes.io/projected/00355fc0-7beb-4357-8aec-62cec0b609ad-kube-api-access-bn75p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.256832 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.256818 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.357814 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.357721 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.357814 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.357773 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn75p\" (UniqueName: \"kubernetes.io/projected/00355fc0-7beb-4357-8aec-62cec0b609ad-kube-api-access-bn75p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.358049 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.357824 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.358049 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.357844 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.358049 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.357868 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00355fc0-7beb-4357-8aec-62cec0b609ad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.358049 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.357895 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.358252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.358227 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.361811 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.358619 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.361811 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.358627 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.364809 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.364309 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00355fc0-7beb-4357-8aec-62cec0b609ad-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.366033 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.366007 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00355fc0-7beb-4357-8aec-62cec0b609ad-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.366953 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.366929 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn75p\" (UniqueName: \"kubernetes.io/projected/00355fc0-7beb-4357-8aec-62cec0b609ad-kube-api-access-bn75p\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mb4j7\" (UID: \"00355fc0-7beb-4357-8aec-62cec0b609ad\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.418283 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.418254 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:12:54.549002 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.548974 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7"] Aug 04 00:12:54.550814 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:12:54.550770 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00355fc0_7beb_4357_8aec_62cec0b609ad.slice/crio-730919f4a661c336cb23b340f104b5489ec0ec6aa6be9fa065cff82f56d970c3 WatchSource:0}: Error finding container 730919f4a661c336cb23b340f104b5489ec0ec6aa6be9fa065cff82f56d970c3: Status 404 returned error can't find the container with id 730919f4a661c336cb23b340f104b5489ec0ec6aa6be9fa065cff82f56d970c3 Aug 04 00:12:54.630010 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:54.629979 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" event={"ID":"00355fc0-7beb-4357-8aec-62cec0b609ad","Type":"ContainerStarted","Data":"730919f4a661c336cb23b340f104b5489ec0ec6aa6be9fa065cff82f56d970c3"} Aug 04 00:12:55.637496 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:55.637455 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" event={"ID":"00355fc0-7beb-4357-8aec-62cec0b609ad","Type":"ContainerStarted","Data":"6e2d781261e2c29ec033d5882d9f78ebd5effac397e4fbef107a491ed85ec324"} Aug 04 00:12:55.656015 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:55.655968 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" podStartSLOduration=1.475202105 podStartE2EDuration="1.655952365s" totalImagesPullingTime="180.75026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:12:55.655172198 +0000 UTC m=+641.535619736" watchObservedRunningTime="2026-08-04 00:12:55.655952365 +0000 UTC m=+641.536399905" Aug 04 00:12:56.324166 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.324135 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:12:56.599711 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.599625 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf"] Aug 04 00:12:56.603300 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.603279 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.606066 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.606046 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Aug 04 00:12:56.612257 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.612234 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf"] Aug 04 00:12:56.629395 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.629368 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:12:56.676289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.676242 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.676751 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.676376 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kszmv\" (UniqueName: \"kubernetes.io/projected/48efe6d7-3773-4953-8c70-493efb11aa65-kube-api-access-kszmv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.676751 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.676422 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48efe6d7-3773-4953-8c70-493efb11aa65-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.676751 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.676525 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.676751 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.676580 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.676751 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.676623 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778011 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.777972 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778198 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778073 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778198 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778122 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszmv\" (UniqueName: \"kubernetes.io/projected/48efe6d7-3773-4953-8c70-493efb11aa65-kube-api-access-kszmv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778198 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778170 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48efe6d7-3773-4953-8c70-493efb11aa65-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778382 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778211 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778382 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778259 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778584 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778551 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778754 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778724 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.778962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.778945 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.781008 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.780975 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48efe6d7-3773-4953-8c70-493efb11aa65-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.781262 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.781241 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48efe6d7-3773-4953-8c70-493efb11aa65-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.791326 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.791300 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kszmv\" (UniqueName: \"kubernetes.io/projected/48efe6d7-3773-4953-8c70-493efb11aa65-kube-api-access-kszmv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf\" (UID: \"48efe6d7-3773-4953-8c70-493efb11aa65\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:56.913995 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:56.913960 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:12:57.075678 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:57.075643 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf"] Aug 04 00:12:57.075784 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:12:57.075719 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48efe6d7_3773_4953_8c70_493efb11aa65.slice/crio-a0105bef2f9c465030ad2a6894c545b845bbfc06c3dc969db51d44a9419c00e7 WatchSource:0}: Error finding container a0105bef2f9c465030ad2a6894c545b845bbfc06c3dc969db51d44a9419c00e7: Status 404 returned error can't find the container with id a0105bef2f9c465030ad2a6894c545b845bbfc06c3dc969db51d44a9419c00e7 Aug 04 00:12:57.649865 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:57.649824 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" event={"ID":"48efe6d7-3773-4953-8c70-493efb11aa65","Type":"ContainerStarted","Data":"d18c887b03d61dd83f847ff42581aeb52513f5f8623389e41438f7ea2e73bce4"} Aug 04 00:12:57.649865 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:57.649867 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" event={"ID":"48efe6d7-3773-4953-8c70-493efb11aa65","Type":"ContainerStarted","Data":"a0105bef2f9c465030ad2a6894c545b845bbfc06c3dc969db51d44a9419c00e7"} Aug 04 00:12:57.674334 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:12:57.669887 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" podStartSLOduration=1.508448955 podStartE2EDuration="1.669870062s" totalImagesPullingTime="161.421107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:12:57.666626431 +0000 UTC m=+643.547073971" watchObservedRunningTime="2026-08-04 00:12:57.669870062 +0000 UTC m=+643.550317604" Aug 04 00:13:01.423198 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:01.423133 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:13:01.516777 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:01.516738 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:13:04.418887 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:04.418854 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:13:04.419327 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:04.418911 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:13:04.431181 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:04.431156 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:13:04.699714 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:04.699623 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mb4j7" Aug 04 00:13:04.824435 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:04.824400 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:13:06.914263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:06.914231 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:13:06.914639 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:06.914280 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:13:06.926407 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:06.926382 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:13:07.721452 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:07.721423 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf" Aug 04 00:13:09.951293 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:13:09.951242 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:14:07.178258 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:07.178163 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:14:07.279622 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:07.279590 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:14:08.327302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:08.327267 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:14:08.426426 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:08.426392 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:14:35.934748 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:35.934699 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:14:45.922476 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:45.922446 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:14:55.061276 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.061235 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-89565659c-wh97j"] Aug 04 00:14:55.061705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.061460 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-89565659c-wh97j" podUID="fe048780-e1ae-4b93-bcff-6757f660307e" containerName="manager" containerID="cri-o://61252794aa7b17275b2991796bf049fca98e588bdb365829cd88023196556754" gracePeriod=30 Aug 04 00:14:55.294004 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.293970 2551 generic.go:356] "Generic (PLEG): container finished" podID="fe048780-e1ae-4b93-bcff-6757f660307e" containerID="61252794aa7b17275b2991796bf049fca98e588bdb365829cd88023196556754" exitCode=0 Aug 04 00:14:55.294116 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.294034 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89565659c-wh97j" event={"ID":"fe048780-e1ae-4b93-bcff-6757f660307e","Type":"ContainerDied","Data":"61252794aa7b17275b2991796bf049fca98e588bdb365829cd88023196556754"} Aug 04 00:14:55.315499 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.315447 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:14:55.413111 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.413077 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe048780-e1ae-4b93-bcff-6757f660307e-webhook-cert\") pod \"fe048780-e1ae-4b93-bcff-6757f660307e\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " Aug 04 00:14:55.413291 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.413209 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftwk2\" (UniqueName: \"kubernetes.io/projected/fe048780-e1ae-4b93-bcff-6757f660307e-kube-api-access-ftwk2\") pod \"fe048780-e1ae-4b93-bcff-6757f660307e\" (UID: \"fe048780-e1ae-4b93-bcff-6757f660307e\") " Aug 04 00:14:55.415055 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.415027 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe048780-e1ae-4b93-bcff-6757f660307e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fe048780-e1ae-4b93-bcff-6757f660307e" (UID: "fe048780-e1ae-4b93-bcff-6757f660307e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:14:55.415191 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.415100 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe048780-e1ae-4b93-bcff-6757f660307e-kube-api-access-ftwk2" (OuterVolumeSpecName: "kube-api-access-ftwk2") pod "fe048780-e1ae-4b93-bcff-6757f660307e" (UID: "fe048780-e1ae-4b93-bcff-6757f660307e"). InnerVolumeSpecName "kube-api-access-ftwk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:14:55.514780 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.514744 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftwk2\" (UniqueName: \"kubernetes.io/projected/fe048780-e1ae-4b93-bcff-6757f660307e-kube-api-access-ftwk2\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:14:55.514780 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:55.514776 2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe048780-e1ae-4b93-bcff-6757f660307e-webhook-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:14:56.301762 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:56.301732 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89565659c-wh97j" Aug 04 00:14:56.301762 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:56.301740 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89565659c-wh97j" event={"ID":"fe048780-e1ae-4b93-bcff-6757f660307e","Type":"ContainerDied","Data":"7ef6153d006def962fe4ebba40d24e293ba57acc4e17dae606448486fbc8b6de"} Aug 04 00:14:56.302229 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:56.301817 2551 scope.go:117] "RemoveContainer" containerID="61252794aa7b17275b2991796bf049fca98e588bdb365829cd88023196556754" Aug 04 00:14:56.323619 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:56.323597 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-89565659c-wh97j"] Aug 04 00:14:56.329109 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:56.329088 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-89565659c-wh97j"] Aug 04 00:14:56.658634 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:56.658602 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe048780-e1ae-4b93-bcff-6757f660307e" path="/var/lib/kubelet/pods/fe048780-e1ae-4b93-bcff-6757f660307e/volumes" Aug 04 00:14:58.425372 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:14:58.425332 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:16:15.396949 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:16:15.396916 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:16:16.584105 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:16:16.584064 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:16:40.224670 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:16:40.224638 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:16:50.021051 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:16:50.021012 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:17:19.402880 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:19.402835 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:19.530172 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:19.530144 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:19.702068 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:19.701986 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:23.599396 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:23.599355 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:23.807300 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:23.807264 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:23.857159 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:23.857071 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:25.284848 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:25.284808 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:25.352951 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:25.352916 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:25.485827 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:25.485763 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:25.632456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:25.632419 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:25.732385 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:25.732355 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:31.422914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:31.422879 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:32.883778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:32.883740 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:17:46.926660 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:46.926625 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:17:54.328145 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:17:54.328107 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:18:09.627910 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:18:09.627875 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:18:15.819220 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:18:15.819185 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:18:25.022299 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:18:25.022254 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:18:32.431771 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:18:32.431735 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:18:45.619003 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:18:45.618964 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:18:50.727651 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:18:50.727619 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:18:59.422628 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:18:59.422588 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:19:08.021634 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:19:08.021600 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:19:20.635867 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:19:20.635833 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:19:31.125270 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:19:31.125232 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:19:53.934047 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:19:53.934008 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:04.442712 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:04.442675 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:10.126493 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:10.126413 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:17.819811 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:17.819757 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:25.017344 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:25.017308 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:30.128978 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:30.128941 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:35.625272 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:35.625238 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:43.216081 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:43.216048 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:20:54.848232 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:54.848198 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:20:54.926729 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:20:54.926697 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:21:02.424949 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:02.424913 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:21:13.022911 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:13.022873 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:21:18.223046 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:18.223007 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:21:21.963255 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:21.963212 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:21:24.430068 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:24.430035 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:21:33.429976 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:33.429931 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:21:38.727554 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:38.727512 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:21:49.232189 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:21:49.232149 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:22:44.608161 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.608123 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl"] Aug 04 00:22:44.610641 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.608350 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" podUID="e7c636de-b22b-4a68-95d6-37a448ccd073" containerName="manager" containerID="cri-o://7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8" gracePeriod=10 Aug 04 00:22:44.864137 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.864085 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:22:44.868308 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.868284 2551 generic.go:356] "Generic (PLEG): container finished" podID="e7c636de-b22b-4a68-95d6-37a448ccd073" containerID="7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8" exitCode=0 Aug 04 00:22:44.868427 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.868322 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" Aug 04 00:22:44.868427 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.868362 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" event={"ID":"e7c636de-b22b-4a68-95d6-37a448ccd073","Type":"ContainerDied","Data":"7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8"} Aug 04 00:22:44.868427 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.868422 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl" event={"ID":"e7c636de-b22b-4a68-95d6-37a448ccd073","Type":"ContainerDied","Data":"a3bfddeec9321443c3a7cd13faaf9c97d1676099f76447a68f7aa3df31e2612e"} Aug 04 00:22:44.868533 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.868445 2551 scope.go:117] "RemoveContainer" containerID="7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8" Aug 04 00:22:44.877560 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.877543 2551 scope.go:117] "RemoveContainer" containerID="7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8" Aug 04 00:22:44.877786 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:22:44.877769 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8\": container with ID starting with 7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8 not found: ID does not exist" containerID="7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8" Aug 04 00:22:44.877849 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.877808 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8"} err="failed to get container status \"7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8\": rpc error: code = NotFound desc = could not find container \"7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8\": container with ID starting with 7c7f14e53428926d477e4e0c4a7f0cf17ed2594a294e69084fc3db4f770fbfa8 not found: ID does not exist" Aug 04 00:22:44.988171 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.988135 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c636de-b22b-4a68-95d6-37a448ccd073-extensions-socket-volume\") pod \"e7c636de-b22b-4a68-95d6-37a448ccd073\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " Aug 04 00:22:44.988341 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.988181 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp8nb\" (UniqueName: \"kubernetes.io/projected/e7c636de-b22b-4a68-95d6-37a448ccd073-kube-api-access-fp8nb\") pod \"e7c636de-b22b-4a68-95d6-37a448ccd073\" (UID: \"e7c636de-b22b-4a68-95d6-37a448ccd073\") " Aug 04 00:22:44.988507 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.988484 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7c636de-b22b-4a68-95d6-37a448ccd073-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7c636de-b22b-4a68-95d6-37a448ccd073" (UID: "e7c636de-b22b-4a68-95d6-37a448ccd073"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:22:44.990292 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:44.990264 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7c636de-b22b-4a68-95d6-37a448ccd073-kube-api-access-fp8nb" (OuterVolumeSpecName: "kube-api-access-fp8nb") pod "e7c636de-b22b-4a68-95d6-37a448ccd073" (UID: "e7c636de-b22b-4a68-95d6-37a448ccd073"). InnerVolumeSpecName "kube-api-access-fp8nb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:22:45.089900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:45.089865 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c636de-b22b-4a68-95d6-37a448ccd073-extensions-socket-volume\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:22:45.089900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:45.089893 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp8nb\" (UniqueName: \"kubernetes.io/projected/e7c636de-b22b-4a68-95d6-37a448ccd073-kube-api-access-fp8nb\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:22:45.192294 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:45.192268 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl"] Aug 04 00:22:45.197471 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:45.197451 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-wxwjl"] Aug 04 00:22:46.658545 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:22:46.658509 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7c636de-b22b-4a68-95d6-37a448ccd073" path="/var/lib/kubelet/pods/e7c636de-b22b-4a68-95d6-37a448ccd073/volumes" Aug 04 00:23:45.327466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.327431 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv"] Aug 04 00:23:45.327962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.327814 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe048780-e1ae-4b93-bcff-6757f660307e" containerName="manager" Aug 04 00:23:45.327962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.327826 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe048780-e1ae-4b93-bcff-6757f660307e" containerName="manager" Aug 04 00:23:45.327962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.327842 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7c636de-b22b-4a68-95d6-37a448ccd073" containerName="manager" Aug 04 00:23:45.327962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.327847 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7c636de-b22b-4a68-95d6-37a448ccd073" containerName="manager" Aug 04 00:23:45.327962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.327901 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7c636de-b22b-4a68-95d6-37a448ccd073" containerName="manager" Aug 04 00:23:45.327962 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.327910 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe048780-e1ae-4b93-bcff-6757f660307e" containerName="manager" Aug 04 00:23:45.330925 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.330905 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.333647 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.333627 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mgscw\"" Aug 04 00:23:45.342563 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.342541 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv"] Aug 04 00:23:45.431367 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.431337 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrc2k\" (UniqueName: \"kubernetes.io/projected/700bbe91-a81a-411f-b980-08d457c16e04-kube-api-access-vrc2k\") pod \"kuadrant-operator-controller-manager-68cf589589-8cjcv\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.431512 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.431455 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700bbe91-a81a-411f-b980-08d457c16e04-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8cjcv\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.532589 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.532551 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700bbe91-a81a-411f-b980-08d457c16e04-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8cjcv\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.532753 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.532616 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrc2k\" (UniqueName: \"kubernetes.io/projected/700bbe91-a81a-411f-b980-08d457c16e04-kube-api-access-vrc2k\") pod \"kuadrant-operator-controller-manager-68cf589589-8cjcv\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.533016 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.532991 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700bbe91-a81a-411f-b980-08d457c16e04-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8cjcv\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.541778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.541755 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrc2k\" (UniqueName: \"kubernetes.io/projected/700bbe91-a81a-411f-b980-08d457c16e04-kube-api-access-vrc2k\") pod \"kuadrant-operator-controller-manager-68cf589589-8cjcv\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.641722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.641698 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:45.790348 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:45.790321 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv"] Aug 04 00:23:45.792059 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:23:45.792036 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700bbe91_a81a_411f_b980_08d457c16e04.slice/crio-00ceadb3a32984d6f9c0db2b1a51bc891cd2d739233a6a1d857b3bb947a622a0 WatchSource:0}: Error finding container 00ceadb3a32984d6f9c0db2b1a51bc891cd2d739233a6a1d857b3bb947a622a0: Status 404 returned error can't find the container with id 00ceadb3a32984d6f9c0db2b1a51bc891cd2d739233a6a1d857b3bb947a622a0 Aug 04 00:23:46.207176 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:46.207141 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" event={"ID":"700bbe91-a81a-411f-b980-08d457c16e04","Type":"ContainerStarted","Data":"a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf"} Aug 04 00:23:46.207343 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:46.207202 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:23:46.207343 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:46.207222 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" event={"ID":"700bbe91-a81a-411f-b980-08d457c16e04","Type":"ContainerStarted","Data":"00ceadb3a32984d6f9c0db2b1a51bc891cd2d739233a6a1d857b3bb947a622a0"} Aug 04 00:23:46.231741 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:46.231691 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" podStartSLOduration=1.23167787 podStartE2EDuration="1.23167787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:23:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:23:46.228731016 +0000 UTC m=+1292.109178556" watchObservedRunningTime="2026-08-04 00:23:46.23167787 +0000 UTC m=+1292.112125409" Aug 04 00:23:57.215511 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:23:57.215477 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:24:06.923939 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:24:06.923903 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:24:13.927493 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:24:13.927452 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:24:35.626389 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:24:35.626351 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:24:52.925200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:24:52.925166 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:01.825260 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:01.825225 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:03.584200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:03.584161 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:25:06.826940 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:06.826907 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:14.029116 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:14.029078 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:21.519715 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:21.519676 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:27.627822 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:27.627776 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:36.730353 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:36.730317 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:43.923824 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:43.923775 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:51.823539 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:51.823499 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:25:58.127912 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:25:58.127879 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:26:06.918265 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:26:06.918175 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:26:31.832914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:26:31.832875 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:26:36.921740 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:26:36.921702 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:26:38.437629 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:26:38.437595 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:26:38.523806 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:26:38.523761 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:26:47.932889 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:26:47.932855 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:26:50.728779 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:26:50.728743 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:27:04.026390 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:04.026354 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:27:06.525014 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:06.524979 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:27:29.592414 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:29.592382 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:27:30.319416 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:30.319362 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:27:32.117996 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:32.117954 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:27:40.921625 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:40.921591 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:27:45.029980 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:45.029949 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:27:52.428618 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:27:52.428571 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:02.223701 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:02.223652 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:09.530903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:09.530860 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:16.225342 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:16.225307 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:24.617959 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:24.617920 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:36.680392 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:36.680351 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:41.816670 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:41.816636 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:48.126818 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:48.126770 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:53.420184 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:53.420149 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:28:59.427598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:28:59.427560 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:06.223420 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:06.223382 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:19.830574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:19.830497 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:25.121661 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:25.121622 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:31.332830 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:31.332782 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:36.218536 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:36.218499 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:42.328674 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:42.328638 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:52.224262 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:52.224227 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:29:59.429142 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:29:59.429098 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:30:09.225237 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:30:09.225200 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:30:15.433914 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:30:15.433881 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:30:26.029231 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:30:26.029191 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:30:41.322412 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:30:41.322376 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:30:48.031188 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:30:48.031153 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:30:51.120509 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:30:51.120458 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:30:58.225790 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:30:58.225749 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:31:05.524516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:05.524472 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:31:13.818635 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:13.818601 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:31:16.656326 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:16.656283 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:19.587286 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:19.587249 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:19.694819 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:19.690208 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:20.017504 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:20.017464 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:21.431829 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:21.431783 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:31:25.765406 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:25.765365 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:25.865684 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:25.865644 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:25.995386 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:25.995359 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:26.290902 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:26.290863 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:26.559648 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:26.559564 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:29.041184 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:29.041138 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:29.129070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:29.129037 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:30.291642 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:30.291607 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:30.556268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:30.556179 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:30.786301 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:30.786271 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:30.886979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:30.886944 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:31:33.019388 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:31:33.019352 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:32:01.090487 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:01.090446 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:01.179213 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:01.179173 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:01.258976 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:01.258918 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:01.373272 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:01.373234 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:01.565531 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:01.565495 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:01.948668 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:01.948627 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:02.028438 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.028409 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:02.107642 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.107600 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:02.391925 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.391889 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v"] Aug 04 00:32:02.395565 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.395545 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.400019 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.399998 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-41f75963-gw-openshift-default-dockercfg-zjq6z\"" Aug 04 00:32:02.415948 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.415926 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v"] Aug 04 00:32:02.423134 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.423113 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:02.537062 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537029 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-certs\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537243 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537083 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ncsg\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-kube-api-access-2ncsg\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537243 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537149 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-credential-socket\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537243 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537175 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-istio-token\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537243 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537224 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-envoy\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537415 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537256 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-socket\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537415 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537294 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac63c212-037c-4416-a445-681b6813f971-istio-podinfo\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537415 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537343 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac63c212-037c-4416-a445-681b6813f971-istiod-ca-cert\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.537415 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.537374 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-data\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.638540 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638499 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac63c212-037c-4416-a445-681b6813f971-istiod-ca-cert\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.638734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638555 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-data\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.638734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638589 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-certs\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.638734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638642 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ncsg\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-kube-api-access-2ncsg\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.638734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638690 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-credential-socket\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.638734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638717 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-istio-token\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639182 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638756 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-envoy\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639182 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638884 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-socket\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639182 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.638918 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac63c212-037c-4416-a445-681b6813f971-istio-podinfo\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639182 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.639026 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-data\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639387 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.639180 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-credential-socket\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639387 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.639307 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac63c212-037c-4416-a445-681b6813f971-istiod-ca-cert\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639472 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.639395 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-certs\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.639578 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.639557 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-socket\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.641345 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.641312 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-envoy\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.641493 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.641472 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac63c212-037c-4416-a445-681b6813f971-istio-podinfo\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.654416 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.654360 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ncsg\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-kube-api-access-2ncsg\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.657086 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.657062 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-istio-token\") pod \"e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.705894 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.705870 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:02.850826 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.850783 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v"] Aug 04 00:32:02.852825 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:32:02.852776 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac63c212_037c_4416_a445_681b6813f971.slice/crio-1d20d4e9332b1fdf5908f0701a2612eafe3ea444956ae52923a99057518f560f WatchSource:0}: Error finding container 1d20d4e9332b1fdf5908f0701a2612eafe3ea444956ae52923a99057518f560f: Status 404 returned error can't find the container with id 1d20d4e9332b1fdf5908f0701a2612eafe3ea444956ae52923a99057518f560f Aug 04 00:32:02.855107 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.855075 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:32:02.855180 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.855144 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:32:02.855180 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.855173 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:32:02.926317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.926286 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:02.948520 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.948496 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" event={"ID":"ac63c212-037c-4416-a445-681b6813f971","Type":"ContainerStarted","Data":"e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b"} Aug 04 00:32:02.948647 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.948542 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" event={"ID":"ac63c212-037c-4416-a445-681b6813f971","Type":"ContainerStarted","Data":"1d20d4e9332b1fdf5908f0701a2612eafe3ea444956ae52923a99057518f560f"} Aug 04 00:32:02.974224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:02.974164 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podStartSLOduration=0.974142803 podStartE2EDuration="974.142803ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:32:02.972404251 +0000 UTC m=+1788.852851791" watchObservedRunningTime="2026-08-04 00:32:02.974142803 +0000 UTC m=+1788.854590345" Aug 04 00:32:03.706532 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:03.706495 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:03.711388 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:03.711365 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:03.923568 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:03.923531 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:32:03.957209 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:03.957124 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:03.960067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:03.960043 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:04.432554 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:04.432525 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:04.538122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:04.538085 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:04.921599 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:04.921567 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:05.058232 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:05.058195 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:05.133806 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:05.133774 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:11.543078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:11.542995 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:32:17.036365 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:17.036279 2551 generic.go:356] "Generic (PLEG): container finished" podID="ac63c212-037c-4416-a445-681b6813f971" containerID="e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b" exitCode=0 Aug 04 00:32:17.036365 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:17.036320 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" event={"ID":"ac63c212-037c-4416-a445-681b6813f971","Type":"ContainerDied","Data":"e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b"} Aug 04 00:32:17.036959 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:17.036788 2551 scope.go:117] "RemoveContainer" containerID="e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b" Aug 04 00:32:17.037823 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:17.037778 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:32:17.037923 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:17.037858 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:32:17.037923 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:17.037892 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:32:18.043747 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:18.043709 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" event={"ID":"ac63c212-037c-4416-a445-681b6813f971","Type":"ContainerStarted","Data":"ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61"} Aug 04 00:32:18.706038 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:18.706003 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:18.707177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:18.707149 2551 patch_prober.go:28] interesting pod/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Aug 04 00:32:18.707295 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:18.707198 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Aug 04 00:32:18.957960 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:18.957868 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:19.707116 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:19.707083 2551 patch_prober.go:28] interesting pod/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Aug 04 00:32:19.707516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:19.707142 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Aug 04 00:32:20.707126 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:20.707082 2551 patch_prober.go:28] interesting pod/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Aug 04 00:32:20.707522 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:20.707159 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Aug 04 00:32:21.706968 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:21.706928 2551 patch_prober.go:28] interesting pod/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Aug 04 00:32:21.707161 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:21.707002 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Aug 04 00:32:22.706935 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:22.706901 2551 patch_prober.go:28] interesting pod/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Aug 04 00:32:22.707089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:22.706957 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Aug 04 00:32:23.710618 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:23.710584 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:23.711589 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:23.711569 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:35.977765 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:35.977725 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:36.045556 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.045524 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:36.107059 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.107016 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:36.271429 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.271336 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v"] Aug 04 00:32:36.271669 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.271616 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" containerID="cri-o://ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61" gracePeriod=30 Aug 04 00:32:36.392151 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.392098 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:36.444936 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.444904 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:36.516601 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.516512 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:36.736554 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.736519 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:36.793627 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:36.793538 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:37.077284 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:37.077182 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:37.177476 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:37.177429 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:38.713091 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:38.713056 2551 patch_prober.go:28] interesting pod/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 00:32:38.713448 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:38.713126 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 00:32:39.020499 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:39.020419 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:32:39.410778 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:39.410736 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:39.511666 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:39.511630 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:40.530346 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:40.530310 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:40.730188 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:40.730152 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:41.132086 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.132045 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:41.231993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.231955 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:32:41.529890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.529867 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:41.612769 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.612738 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-istio-token\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.612769 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.612775 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-certs\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.612823 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-data\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.612875 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-socket\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.612925 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-envoy\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.612951 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac63c212-037c-4416-a445-681b6813f971-istio-podinfo\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.612977 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac63c212-037c-4416-a445-681b6813f971-istiod-ca-cert\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613011 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-credential-socket\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613042 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613158 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ncsg\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-kube-api-access-2ncsg\") pod \"ac63c212-037c-4416-a445-681b6813f971\" (UID: \"ac63c212-037c-4416-a445-681b6813f971\") " Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613182 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:32:41.613253 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613226 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:32:41.613707 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613473 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-certs\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.613707 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613494 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-data\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.613707 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613507 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-workload-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.613707 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613590 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac63c212-037c-4416-a445-681b6813f971-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:32:41.613707 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.613674 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:32:41.615576 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.615531 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-kube-api-access-2ncsg" (OuterVolumeSpecName: "kube-api-access-2ncsg") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "kube-api-access-2ncsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:32:41.615576 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.615526 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:32:41.615878 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.615856 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:32:41.615946 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.615895 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac63c212-037c-4416-a445-681b6813f971-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac63c212-037c-4416-a445-681b6813f971" (UID: "ac63c212-037c-4416-a445-681b6813f971"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 00:32:41.714089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.714011 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ncsg\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-kube-api-access-2ncsg\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.714089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.714040 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac63c212-037c-4416-a445-681b6813f971-istio-token\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.714089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.714050 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-istio-envoy\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.714089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.714057 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac63c212-037c-4416-a445-681b6813f971-istio-podinfo\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.714089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.714066 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac63c212-037c-4416-a445-681b6813f971-istiod-ca-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:41.714089 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:41.714074 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac63c212-037c-4416-a445-681b6813f971-credential-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:32:42.178723 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.178683 2551 generic.go:356] "Generic (PLEG): container finished" podID="ac63c212-037c-4416-a445-681b6813f971" containerID="ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61" exitCode=0 Aug 04 00:32:42.178942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.178736 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" Aug 04 00:32:42.178942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.178773 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" event={"ID":"ac63c212-037c-4416-a445-681b6813f971","Type":"ContainerDied","Data":"ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61"} Aug 04 00:32:42.178942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.178844 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v" event={"ID":"ac63c212-037c-4416-a445-681b6813f971","Type":"ContainerDied","Data":"1d20d4e9332b1fdf5908f0701a2612eafe3ea444956ae52923a99057518f560f"} Aug 04 00:32:42.178942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.178866 2551 scope.go:117] "RemoveContainer" containerID="ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61" Aug 04 00:32:42.188291 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.188280 2551 scope.go:117] "RemoveContainer" containerID="e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b" Aug 04 00:32:42.196317 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.196301 2551 scope.go:117] "RemoveContainer" containerID="ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61" Aug 04 00:32:42.196560 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:32:42.196540 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61\": container with ID starting with ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61 not found: ID does not exist" containerID="ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61" Aug 04 00:32:42.196611 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.196569 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61"} err="failed to get container status \"ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61\": rpc error: code = NotFound desc = could not find container \"ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61\": container with ID starting with ffa567ff36f2c9bb80c9a50d831d4e4278c36e8135efb07f2687523eae88ef61 not found: ID does not exist" Aug 04 00:32:42.196611 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.196586 2551 scope.go:117] "RemoveContainer" containerID="e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b" Aug 04 00:32:42.196825 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:32:42.196810 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b\": container with ID starting with e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b not found: ID does not exist" containerID="e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b" Aug 04 00:32:42.196871 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.196828 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b"} err="failed to get container status \"e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b\": rpc error: code = NotFound desc = could not find container \"e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b\": container with ID starting with e0dd0f7dffec389576be031f6745d8d84735666228f0d13eb2295bfcb60b631b not found: ID does not exist" Aug 04 00:32:42.200915 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.200893 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v"] Aug 04 00:32:42.204825 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.204784 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-41f75963-gw-openshift-default-797c68b5b-w5s2v"] Aug 04 00:32:42.659912 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:42.659876 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac63c212-037c-4416-a445-681b6813f971" path="/var/lib/kubelet/pods/ac63c212-037c-4416-a445-681b6813f971/volumes" Aug 04 00:32:47.235136 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:32:47.235101 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:33:11.005312 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:11.005281 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:11.071671 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:11.071626 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:11.152332 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:11.152297 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:11.205439 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:11.205401 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:12.269067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.269034 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:12.473235 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.473190 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:12.685755 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.685718 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:12.964537 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.964455 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7"] Aug 04 00:33:12.964982 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.964957 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" Aug 04 00:33:12.964982 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.964980 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" Aug 04 00:33:12.965233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.965010 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" Aug 04 00:33:12.965233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.965018 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" Aug 04 00:33:12.965233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.965115 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" Aug 04 00:33:12.965233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.965130 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac63c212-037c-4416-a445-681b6813f971" containerName="istio-proxy" Aug 04 00:33:12.968448 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.968428 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:12.970935 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.970918 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-59a98a02-openshift-default-dockercfg-x8cw5\"" Aug 04 00:33:12.980632 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.980608 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7"] Aug 04 00:33:12.993295 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:12.993265 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:13.002401 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002376 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-socket\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002532 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002413 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a264d6-88c3-4ba1-b506-fcede9138d21-istiod-ca-cert\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002532 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002454 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6lt2\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-kube-api-access-w6lt2\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002532 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002501 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-token\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002553 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-data\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002587 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-podinfo\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002628 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-envoy\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002658 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-certs\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.002722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.002694 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-credential-socket\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.103936 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.103905 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-envoy\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104119 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.103949 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-certs\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104119 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.103986 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-credential-socket\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104119 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104009 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-socket\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104119 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104026 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a264d6-88c3-4ba1-b506-fcede9138d21-istiod-ca-cert\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104119 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104055 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6lt2\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-kube-api-access-w6lt2\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104119 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104113 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-token\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104429 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104153 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-data\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104429 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104185 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-podinfo\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104533 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104507 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-socket\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104533 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104512 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-certs\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104640 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104618 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-credential-socket\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104767 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104742 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-data\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.104908 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.104786 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a264d6-88c3-4ba1-b506-fcede9138d21-istiod-ca-cert\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.105730 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.105702 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:13.106474 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.106451 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-envoy\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.106784 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.106765 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-podinfo\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.111508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.111488 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-token\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.111718 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.111699 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6lt2\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-kube-api-access-w6lt2\") pod \"e2e-derive-59a98a02-openshift-default-76d984576-4cld7\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.282270 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.282181 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:13.428350 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.428321 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7"] Aug 04 00:33:13.429346 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:33:13.429321 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a264d6_88c3_4ba1_b506_fcede9138d21.slice/crio-2cb82ab34c19639326540750063a78217a11c9fcd4b8718427bdac7ccc84f666 WatchSource:0}: Error finding container 2cb82ab34c19639326540750063a78217a11c9fcd4b8718427bdac7ccc84f666: Status 404 returned error can't find the container with id 2cb82ab34c19639326540750063a78217a11c9fcd4b8718427bdac7ccc84f666 Aug 04 00:33:13.431531 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.431495 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:33:13.431629 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.431577 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:33:13.431629 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:13.431620 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:33:14.030012 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:14.029975 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:33:14.356191 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:14.356093 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" event={"ID":"01a264d6-88c3-4ba1-b506-fcede9138d21","Type":"ContainerStarted","Data":"9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725"} Aug 04 00:33:14.356191 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:14.356147 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" event={"ID":"01a264d6-88c3-4ba1-b506-fcede9138d21","Type":"ContainerStarted","Data":"2cb82ab34c19639326540750063a78217a11c9fcd4b8718427bdac7ccc84f666"} Aug 04 00:33:14.421266 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:14.421216 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podStartSLOduration=2.421201509 podStartE2EDuration="2.421201509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:33:14.414747012 +0000 UTC m=+1860.295194552" watchObservedRunningTime="2026-08-04 00:33:14.421201509 +0000 UTC m=+1860.301649048" Aug 04 00:33:15.282878 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:15.282846 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:15.287989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:15.287963 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:15.361299 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:15.361275 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:15.362183 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:15.362164 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:15.867592 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:15.867561 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:15.966230 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:15.966193 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:16.999076 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:16.999041 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:17.127993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:17.127962 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:17.490625 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:17.490590 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:17.598743 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:17.598698 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:19.818561 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:19.818529 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:33:29.442354 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:29.442322 2551 generic.go:356] "Generic (PLEG): container finished" podID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerID="9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725" exitCode=0 Aug 04 00:33:29.442897 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:29.442382 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" event={"ID":"01a264d6-88c3-4ba1-b506-fcede9138d21","Type":"ContainerDied","Data":"9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725"} Aug 04 00:33:29.442989 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:29.442900 2551 scope.go:117] "RemoveContainer" containerID="9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725" Aug 04 00:33:29.444122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:29.444084 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:33:29.444223 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:29.444176 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:33:29.444223 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:29.444217 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:33:30.362114 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:30.362069 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:30.449559 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:30.449526 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" event={"ID":"01a264d6-88c3-4ba1-b506-fcede9138d21","Type":"ContainerStarted","Data":"355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b"} Aug 04 00:33:31.283649 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:31.283608 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:31.284998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:31.284962 2551 patch_prober.go:28] interesting pod/e2e-derive-59a98a02-openshift-default-76d984576-4cld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Aug 04 00:33:31.285118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:31.285024 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Aug 04 00:33:32.283312 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:32.283266 2551 patch_prober.go:28] interesting pod/e2e-derive-59a98a02-openshift-default-76d984576-4cld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Aug 04 00:33:32.283845 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:32.283344 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Aug 04 00:33:33.283202 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:33.283166 2551 patch_prober.go:28] interesting pod/e2e-derive-59a98a02-openshift-default-76d984576-4cld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Aug 04 00:33:33.283496 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:33.283467 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Aug 04 00:33:34.283579 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:34.283531 2551 patch_prober.go:28] interesting pod/e2e-derive-59a98a02-openshift-default-76d984576-4cld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Aug 04 00:33:34.284073 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:34.283602 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Aug 04 00:33:35.282861 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:35.282823 2551 patch_prober.go:28] interesting pod/e2e-derive-59a98a02-openshift-default-76d984576-4cld7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Aug 04 00:33:35.283068 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:35.282885 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Aug 04 00:33:36.287395 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:36.287366 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:36.287814 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:36.287671 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:36.288280 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:36.288261 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:47.480510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:47.480451 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:47.546600 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:47.546560 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:47.736086 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:47.735986 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:47.818166 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:47.818131 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:47.903357 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:47.903323 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7"] Aug 04 00:33:47.903572 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:47.903550 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" containerID="cri-o://355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b" gracePeriod=30 Aug 04 00:33:47.919034 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:47.918991 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:48.134789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:48.134746 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:48.612917 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:48.612777 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:48.700272 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:48.700234 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:48.808716 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:48.808680 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:33:50.528010 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:50.527966 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:33:51.290454 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:51.290408 2551 patch_prober.go:28] interesting pod/e2e-derive-59a98a02-openshift-default-76d984576-4cld7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 00:33:51.290621 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:51.290506 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 00:33:53.161541 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.161516 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:53.288007 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.287924 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a264d6-88c3-4ba1-b506-fcede9138d21-istiod-ca-cert\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288007 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.287982 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6lt2\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-kube-api-access-w6lt2\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288226 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288025 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-credential-socket\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288226 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288051 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-token\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288226 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288078 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-certs\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288226 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288162 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-envoy\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288226 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288207 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-podinfo\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288239 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-data\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288271 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-socket\") pod \"01a264d6-88c3-4ba1-b506-fcede9138d21\" (UID: \"01a264d6-88c3-4ba1-b506-fcede9138d21\") " Aug 04 00:33:53.288553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288356 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:33:53.288553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288381 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a264d6-88c3-4ba1-b506-fcede9138d21-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:33:53.288553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288444 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:33:53.288553 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288548 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:33:53.288890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288634 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-data" (OuterVolumeSpecName: "istio-data") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:33:53.288890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288678 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.288890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288695 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a264d6-88c3-4ba1-b506-fcede9138d21-istiod-ca-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.288890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288711 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-credential-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.288890 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.288723 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-workload-certs\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.290364 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.290338 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-token" (OuterVolumeSpecName: "istio-token") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:33:53.290364 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.290351 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-kube-api-access-w6lt2" (OuterVolumeSpecName: "kube-api-access-w6lt2") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "kube-api-access-w6lt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:33:53.290550 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.290530 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:33:53.290614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.290595 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "01a264d6-88c3-4ba1-b506-fcede9138d21" (UID: "01a264d6-88c3-4ba1-b506-fcede9138d21"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 00:33:53.389739 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.389703 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-podinfo\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.389739 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.389729 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-data\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.389739 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.389738 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6lt2\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-kube-api-access-w6lt2\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.389739 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.389749 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-token\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.390013 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.389758 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a264d6-88c3-4ba1-b506-fcede9138d21-istio-envoy\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:33:53.580874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.580776 2551 generic.go:356] "Generic (PLEG): container finished" podID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerID="355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b" exitCode=0 Aug 04 00:33:53.580874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.580844 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" Aug 04 00:33:53.580874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.580859 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" event={"ID":"01a264d6-88c3-4ba1-b506-fcede9138d21","Type":"ContainerDied","Data":"355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b"} Aug 04 00:33:53.581132 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.580921 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7" event={"ID":"01a264d6-88c3-4ba1-b506-fcede9138d21","Type":"ContainerDied","Data":"2cb82ab34c19639326540750063a78217a11c9fcd4b8718427bdac7ccc84f666"} Aug 04 00:33:53.581132 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.580949 2551 scope.go:117] "RemoveContainer" containerID="355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b" Aug 04 00:33:53.590984 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.590966 2551 scope.go:117] "RemoveContainer" containerID="9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725" Aug 04 00:33:53.600632 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.600609 2551 scope.go:117] "RemoveContainer" containerID="355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b" Aug 04 00:33:53.600929 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:33:53.600895 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b\": container with ID starting with 355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b not found: ID does not exist" containerID="355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b" Aug 04 00:33:53.600929 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.600921 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b"} err="failed to get container status \"355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b\": rpc error: code = NotFound desc = could not find container \"355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b\": container with ID starting with 355a8ff01e2b6989bab14ba3d62eba5ff283a10ed6111a00921ccbc43ba2c71b not found: ID does not exist" Aug 04 00:33:53.601063 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.600939 2551 scope.go:117] "RemoveContainer" containerID="9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725" Aug 04 00:33:53.601176 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:33:53.601157 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725\": container with ID starting with 9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725 not found: ID does not exist" containerID="9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725" Aug 04 00:33:53.601216 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.601183 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725"} err="failed to get container status \"9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725\": rpc error: code = NotFound desc = could not find container \"9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725\": container with ID starting with 9233211da624c1a9f4003c35d7c645fc2700a280bfd1d150c92ef67bdbb02725 not found: ID does not exist" Aug 04 00:33:53.603062 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.603042 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7"] Aug 04 00:33:53.607074 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:53.607055 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-59a98a02-openshift-default-76d984576-4cld7"] Aug 04 00:33:54.659329 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:33:54.659297 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" path="/var/lib/kubelet/pods/01a264d6-88c3-4ba1-b506-fcede9138d21/volumes" Aug 04 00:34:00.388355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:00.388320 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:03.323143 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:03.323113 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:34:06.903106 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:06.903065 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:07.007847 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:07.007812 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:07.103869 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:07.103830 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:07.204766 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:07.204680 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:07.601508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:07.601412 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:33.541965 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:33.541922 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:35.089183 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:35.089139 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:39.632042 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:39.632006 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:39.755191 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:39.755143 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:39.850049 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:39.849998 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:34:56.899289 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:34:56.899241 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:02.792913 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:02.792876 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:02.844150 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:02.844114 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:12.202297 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:12.202205 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:15.465480 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:15.465443 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:15.566436 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:15.566401 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:23.209355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:23.209305 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:29.271990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:29.271936 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:29.334380 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:29.334347 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:29.418165 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:29.418125 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:29.518308 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:29.518272 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:29.617742 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:29.617641 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:30.173051 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:30.173012 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:30.270421 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:30.270384 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:30.382431 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:30.382398 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:31.873178 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:31.873148 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:33.325454 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:33.325422 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:35:38.568990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:38.568946 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:38.670859 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:38.670817 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:39.228224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:39.228180 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:40.824318 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:40.824281 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:35:45.384395 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:45.384358 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:45.484565 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:45.484536 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:45.620957 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:45.620919 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:45.987246 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:45.987212 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:46.157060 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:46.157021 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:48.942649 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:48.942611 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:49.040153 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:49.040117 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:50.227224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:50.227183 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:50.646339 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:50.646293 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:50.745313 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:50.745276 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:50.828670 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:50.828632 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:35:52.528498 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:52.528460 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:35:59.045716 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:35:59.045680 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:13.912861 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:13.912781 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:14.589006 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:14.588971 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:14.688151 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:14.688095 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:15.008638 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:15.008604 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:20.662709 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:20.662675 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:20.777356 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:20.777319 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:31.528282 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:31.528241 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:31.628188 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:31.628152 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:31.710672 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:31.710633 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:31.780296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:31.780199 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:32.395632 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:32.395589 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:33.624062 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:33.624023 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:36:37.881555 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:37.881508 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:37.984360 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:37.984321 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:38.157148 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:38.157046 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:38.420362 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:38.420253 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:38.568315 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:38.568265 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:38.655573 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:38.655542 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:38.758071 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:38.757980 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:44.502283 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:44.502247 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:44.608560 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:44.608501 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:44.703402 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:44.703368 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:44.896195 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:44.896158 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:50.672523 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:50.672490 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:36:50.783047 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:36:50.783007 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:37:12.124928 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:12.124897 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:37:14.185711 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:14.185672 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:37:19.019615 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.019579 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv"] Aug 04 00:37:19.020082 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.019778 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" podUID="700bbe91-a81a-411f-b980-08d457c16e04" containerName="manager" containerID="cri-o://a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf" gracePeriod=10 Aug 04 00:37:19.276319 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.276256 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:37:19.365177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.365150 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700bbe91-a81a-411f-b980-08d457c16e04-extensions-socket-volume\") pod \"700bbe91-a81a-411f-b980-08d457c16e04\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " Aug 04 00:37:19.365318 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.365228 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrc2k\" (UniqueName: \"kubernetes.io/projected/700bbe91-a81a-411f-b980-08d457c16e04-kube-api-access-vrc2k\") pod \"700bbe91-a81a-411f-b980-08d457c16e04\" (UID: \"700bbe91-a81a-411f-b980-08d457c16e04\") " Aug 04 00:37:19.365584 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.365562 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/700bbe91-a81a-411f-b980-08d457c16e04-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "700bbe91-a81a-411f-b980-08d457c16e04" (UID: "700bbe91-a81a-411f-b980-08d457c16e04"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:37:19.367221 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.367198 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700bbe91-a81a-411f-b980-08d457c16e04-kube-api-access-vrc2k" (OuterVolumeSpecName: "kube-api-access-vrc2k") pod "700bbe91-a81a-411f-b980-08d457c16e04" (UID: "700bbe91-a81a-411f-b980-08d457c16e04"). InnerVolumeSpecName "kube-api-access-vrc2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:37:19.466189 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.466161 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrc2k\" (UniqueName: \"kubernetes.io/projected/700bbe91-a81a-411f-b980-08d457c16e04-kube-api-access-vrc2k\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:37:19.466189 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.466185 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/700bbe91-a81a-411f-b980-08d457c16e04-extensions-socket-volume\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:37:19.728408 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.728373 2551 generic.go:356] "Generic (PLEG): container finished" podID="700bbe91-a81a-411f-b980-08d457c16e04" containerID="a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf" exitCode=0 Aug 04 00:37:19.728574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.728416 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" event={"ID":"700bbe91-a81a-411f-b980-08d457c16e04","Type":"ContainerDied","Data":"a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf"} Aug 04 00:37:19.728574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.728428 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" Aug 04 00:37:19.728574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.728459 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv" event={"ID":"700bbe91-a81a-411f-b980-08d457c16e04","Type":"ContainerDied","Data":"00ceadb3a32984d6f9c0db2b1a51bc891cd2d739233a6a1d857b3bb947a622a0"} Aug 04 00:37:19.728574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.728481 2551 scope.go:117] "RemoveContainer" containerID="a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf" Aug 04 00:37:19.738786 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.738768 2551 scope.go:117] "RemoveContainer" containerID="a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf" Aug 04 00:37:19.739099 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:37:19.739071 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf\": container with ID starting with a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf not found: ID does not exist" containerID="a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf" Aug 04 00:37:19.739192 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.739103 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf"} err="failed to get container status \"a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf\": rpc error: code = NotFound desc = could not find container \"a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf\": container with ID starting with a7df5b4f853abae3f63c035ff979652778cdebc02651d56ff272329b3cb494bf not found: ID does not exist" Aug 04 00:37:19.751377 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.751355 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv"] Aug 04 00:37:19.756917 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:19.756894 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8cjcv"] Aug 04 00:37:20.658983 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:37:20.658949 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="700bbe91-a81a-411f-b980-08d457c16e04" path="/var/lib/kubelet/pods/700bbe91-a81a-411f-b980-08d457c16e04/volumes" Aug 04 00:38:34.620677 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.620635 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg"] Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621183 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="700bbe91-a81a-411f-b980-08d457c16e04" containerName="manager" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621202 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="700bbe91-a81a-411f-b980-08d457c16e04" containerName="manager" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621220 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621225 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621242 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621248 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621304 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621312 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="700bbe91-a81a-411f-b980-08d457c16e04" containerName="manager" Aug 04 00:38:34.623252 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.621319 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a264d6-88c3-4ba1-b506-fcede9138d21" containerName="istio-proxy" Aug 04 00:38:34.624186 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.624167 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:34.630541 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.630279 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mgscw\"" Aug 04 00:38:34.632757 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.632735 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8nkb\" (UniqueName: \"kubernetes.io/projected/9b2ce2d1-fe88-473c-a620-b5cd493fc65a-kube-api-access-f8nkb\") pod \"kuadrant-operator-controller-manager-68cf589589-8gqtg\" (UID: \"9b2ce2d1-fe88-473c-a620-b5cd493fc65a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:34.632903 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.632807 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b2ce2d1-fe88-473c-a620-b5cd493fc65a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8gqtg\" (UID: \"9b2ce2d1-fe88-473c-a620-b5cd493fc65a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:34.638040 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.638017 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg"] Aug 04 00:38:34.733865 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.733832 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8nkb\" (UniqueName: \"kubernetes.io/projected/9b2ce2d1-fe88-473c-a620-b5cd493fc65a-kube-api-access-f8nkb\") pod \"kuadrant-operator-controller-manager-68cf589589-8gqtg\" (UID: \"9b2ce2d1-fe88-473c-a620-b5cd493fc65a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:34.734027 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.733891 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b2ce2d1-fe88-473c-a620-b5cd493fc65a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8gqtg\" (UID: \"9b2ce2d1-fe88-473c-a620-b5cd493fc65a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:34.734258 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.734239 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b2ce2d1-fe88-473c-a620-b5cd493fc65a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8gqtg\" (UID: \"9b2ce2d1-fe88-473c-a620-b5cd493fc65a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:34.745326 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.745305 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8nkb\" (UniqueName: \"kubernetes.io/projected/9b2ce2d1-fe88-473c-a620-b5cd493fc65a-kube-api-access-f8nkb\") pod \"kuadrant-operator-controller-manager-68cf589589-8gqtg\" (UID: \"9b2ce2d1-fe88-473c-a620-b5cd493fc65a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:34.933940 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:34.933910 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:35.076233 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:35.076205 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg"] Aug 04 00:38:35.077155 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:38:35.077130 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b2ce2d1_fe88_473c_a620_b5cd493fc65a.slice/crio-e3f6c2b83ad4051d9b25118b58e9abb181584335a53291eb1612084140698a39 WatchSource:0}: Error finding container e3f6c2b83ad4051d9b25118b58e9abb181584335a53291eb1612084140698a39: Status 404 returned error can't find the container with id e3f6c2b83ad4051d9b25118b58e9abb181584335a53291eb1612084140698a39 Aug 04 00:38:35.147443 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:35.147418 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" event={"ID":"9b2ce2d1-fe88-473c-a620-b5cd493fc65a","Type":"ContainerStarted","Data":"e3f6c2b83ad4051d9b25118b58e9abb181584335a53291eb1612084140698a39"} Aug 04 00:38:36.154069 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:36.154035 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" event={"ID":"9b2ce2d1-fe88-473c-a620-b5cd493fc65a","Type":"ContainerStarted","Data":"888f1ad440d94e1e1dcef0c92d5123ff3534aa182d4ff53a960cd064f5f6b618"} Aug 04 00:38:36.154504 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:36.154183 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:36.196683 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:36.196625 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" podStartSLOduration=2.196608293 podStartE2EDuration="2.196608293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:38:36.195302114 +0000 UTC m=+2182.075749656" watchObservedRunningTime="2026-08-04 00:38:36.196608293 +0000 UTC m=+2182.077055834" Aug 04 00:38:47.161578 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:47.161548 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8gqtg" Aug 04 00:38:53.616733 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:53.616698 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:53.888274 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:53.888244 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:53.987205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:53.987169 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:56.908482 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:56.908451 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:57.009526 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:57.009496 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:57.027980 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:57.027946 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:38:58.041510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:58.041465 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:58.384637 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:58.384607 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:58.691421 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:58.691333 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:38:58.787084 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:38:58.787048 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:39:04.823205 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:39:04.823150 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:39:21.877408 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:39:21.877356 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:39:30.235067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:39:30.235031 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:39:32.528683 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:39:32.528640 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:39:45.730259 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:39:45.730225 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:40:02.029208 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:02.029161 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:04.163066 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:04.163028 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:04.224139 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:04.224104 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:09.208598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:09.208560 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:40:11.646603 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:11.646556 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:11.716567 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:11.716507 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:11.783287 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:11.783242 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:11.843837 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:11.843786 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:14.771510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:14.771472 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:40:14.888406 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:14.888368 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:19.968478 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:19.968441 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:56.033324 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:56.033287 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:40:56.792358 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:40:56.792317 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:02.532239 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:02.532192 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:02.632453 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:02.632412 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:02.801475 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:02.801375 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:03.028382 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:03.028351 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:04.731693 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:04.731655 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:41:14.277456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:14.277358 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:14.379665 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:14.379629 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:14.480544 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:14.480502 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:15.079896 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:15.079860 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:15.179859 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:15.179828 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:15.280221 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:15.280169 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:16.539268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:16.539236 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:16.921387 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:16.921358 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:41:22.299231 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:22.299178 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:22.399348 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:22.399309 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:22.914054 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:22.914016 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:24.424649 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:24.424615 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:41:29.011328 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:29.011296 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:29.114927 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:29.114873 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:29.211352 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:29.211324 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:39.694527 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:39.694491 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:41:39.917626 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:41:39.917578 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:00.847400 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:00.847339 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:06.615251 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:06.615218 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:06.717355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:06.717324 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:06.898615 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:06.898581 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:07.192480 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:07.192376 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:07.495584 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:07.495495 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:09.938303 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:09.938263 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:10.037864 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:10.037829 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:11.093052 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:11.093016 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:11.279720 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:11.279690 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:11.623463 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:11.623433 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:11.706614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:11.706581 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:13.740791 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:13.740758 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:42:34.915010 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:34.914976 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:35.247347 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.247268 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:35.506298 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.506214 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m"] Aug 04 00:42:35.510247 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.510226 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.513709 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.513689 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5d4def-openshift-default-dockercfg-2qzvm\"" Aug 04 00:42:35.523182 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.523161 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m"] Aug 04 00:42:35.532491 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.532466 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:35.658993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.658962 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fw2x\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-kube-api-access-8fw2x\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.658993 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.658999 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-socket\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.659217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.659024 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-podinfo\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.659217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.659044 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-certs\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.659217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.659061 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-data\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.659217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.659098 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-token\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.659217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.659142 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-credential-socket\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.659217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.659160 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istiod-ca-cert\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.659217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.659182 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-envoy\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.760422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760338 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fw2x\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-kube-api-access-8fw2x\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.760422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760378 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-socket\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.760422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760419 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-podinfo\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.760705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760449 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-certs\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.760705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760471 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-data\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.760705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760578 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-token\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761071 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.761041 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-data\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761071 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760822 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-socket\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.760941 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-certs\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.761148 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-credential-socket\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.761187 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istiod-ca-cert\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.761233 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-envoy\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761495 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.761437 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-credential-socket\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.761998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.761973 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istiod-ca-cert\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.763238 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.763209 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-podinfo\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.763337 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.763282 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-envoy\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.773132 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.773100 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-token\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.773490 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.773469 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fw2x\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-kube-api-access-8fw2x\") pod \"e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.821645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.821606 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:35.967385 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:42:35.967340 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ca720d_6a5f_4cd8_9bd3_307ea417a206.slice/crio-459eafe4d8de572bed19eae96499863359f8d09cc3f7c323366673cfd8dc04c8 WatchSource:0}: Error finding container 459eafe4d8de572bed19eae96499863359f8d09cc3f7c323366673cfd8dc04c8: Status 404 returned error can't find the container with id 459eafe4d8de572bed19eae96499863359f8d09cc3f7c323366673cfd8dc04c8 Aug 04 00:42:35.967737 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.967637 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m"] Aug 04 00:42:35.970032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.969994 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:42:35.970139 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.970072 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:42:35.970139 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:35.970113 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:42:36.251130 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:36.251091 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:36.454406 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:36.454371 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" event={"ID":"b8ca720d-6a5f-4cd8-9bd3-307ea417a206","Type":"ContainerStarted","Data":"638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55"} Aug 04 00:42:36.454611 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:36.454422 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" event={"ID":"b8ca720d-6a5f-4cd8-9bd3-307ea417a206","Type":"ContainerStarted","Data":"459eafe4d8de572bed19eae96499863359f8d09cc3f7c323366673cfd8dc04c8"} Aug 04 00:42:36.474127 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:36.474075 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podStartSLOduration=1.474056353 podStartE2EDuration="1.474056353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:42:36.473431536 +0000 UTC m=+2422.353879081" watchObservedRunningTime="2026-08-04 00:42:36.474056353 +0000 UTC m=+2422.354503896" Aug 04 00:42:36.822551 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:36.822467 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:37.827624 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:37.827596 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:38.380561 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:38.380495 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:38.464535 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:38.464506 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:38.465550 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:38.465531 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:38.485574 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:38.485549 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:39.513875 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:39.513836 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:40.053315 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:40.053275 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:40.135508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:40.135477 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:40.321951 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:40.321871 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:42:42.422770 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:42.422732 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:42:52.547463 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:52.547364 2551 generic.go:356] "Generic (PLEG): container finished" podID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerID="638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55" exitCode=0 Aug 04 00:42:52.547463 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:52.547403 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" event={"ID":"b8ca720d-6a5f-4cd8-9bd3-307ea417a206","Type":"ContainerDied","Data":"638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55"} Aug 04 00:42:52.547935 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:52.547766 2551 scope.go:117] "RemoveContainer" containerID="638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55" Aug 04 00:42:52.548883 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:52.548853 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:42:52.548956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:52.548923 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:42:52.548998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:52.548959 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:42:53.465302 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:53.465274 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:53.555083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:53.555047 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" event={"ID":"b8ca720d-6a5f-4cd8-9bd3-307ea417a206","Type":"ContainerStarted","Data":"9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375"} Aug 04 00:42:53.822054 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:53.821954 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:53.823177 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:53.823150 2551 patch_prober.go:28] interesting pod/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Aug 04 00:42:53.823310 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:53.823205 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Aug 04 00:42:54.822460 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:54.822424 2551 patch_prober.go:28] interesting pod/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Aug 04 00:42:54.822920 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:54.822482 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Aug 04 00:42:55.822378 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:55.822339 2551 patch_prober.go:28] interesting pod/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Aug 04 00:42:55.822582 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:55.822404 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Aug 04 00:42:56.822999 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:56.822961 2551 patch_prober.go:28] interesting pod/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Aug 04 00:42:56.823414 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:56.823018 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Aug 04 00:42:57.822508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:57.822469 2551 patch_prober.go:28] interesting pod/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Aug 04 00:42:57.822676 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:57.822523 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Aug 04 00:42:58.822186 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:58.822142 2551 patch_prober.go:28] interesting pod/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Aug 04 00:42:58.822587 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:58.822213 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Aug 04 00:42:59.826157 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:59.826124 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:59.826551 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:59.826387 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:42:59.826983 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:42:59.826965 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:43:03.336335 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:03.336296 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:43:11.528139 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:11.528106 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:43:13.930187 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:13.930151 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:43:22.025476 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:22.025440 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:43:28.919519 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:28.919484 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:43:32.832919 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:32.832883 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:43:37.035955 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:37.035925 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:43:37.137063 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:43:37.137027 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:07.415095 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:07.415017 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:08.326866 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:08.326831 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:10.827499 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:10.827464 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:44:18.894562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:18.894528 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:18.962845 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:18.962785 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:19.028140 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:19.028099 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:19.087039 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:19.087006 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:19.458220 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:19.458179 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:21.020546 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:21.020509 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:44:25.244196 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:25.244163 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:25.344011 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:25.343977 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:25.471668 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:25.471636 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:25.694270 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:25.694240 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:36.236162 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:36.236129 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:36.302669 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:36.302637 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:36.370475 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:36.370445 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:36.428735 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:36.428693 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:36.772311 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:36.772276 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:38.327386 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:38.327349 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:44:42.642843 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:42.642787 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:42.746248 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:42.746218 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:42.870940 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:42.870906 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m"] Aug 04 00:44:42.871161 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:42.871137 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" containerID="cri-o://9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375" gracePeriod=30 Aug 04 00:44:42.896257 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:42.896141 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:43.121129 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:43.121052 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:43.433478 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:43.433445 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:43.737041 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:43.736928 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:44.027356 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:44.027255 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:44.828727 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:44.828693 2551 patch_prober.go:28] interesting pod/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 00:44:44.829124 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:44.828759 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 00:44:46.869997 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:46.869962 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:46.969416 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:46.969388 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:48.097886 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.097763 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:48.122354 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.122333 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:44:48.174023 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.173949 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-token\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174023 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.173993 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-certs\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174023 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174017 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fw2x\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-kube-api-access-8fw2x\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174034 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-credential-socket\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174054 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-podinfo\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174073 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-data\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174110 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-socket\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174154 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istiod-ca-cert\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174201 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-envoy\") pod \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\" (UID: \"b8ca720d-6a5f-4cd8-9bd3-307ea417a206\") " Aug 04 00:44:48.174277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174227 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:44:48.174598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174387 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:44:48.174598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174404 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-data" (OuterVolumeSpecName: "istio-data") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:44:48.174598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174499 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:44:48.174598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174561 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-certs\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.174598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174578 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-credential-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.174598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174592 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-data\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.174598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.174588 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:44:48.176249 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.176226 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-token" (OuterVolumeSpecName: "istio-token") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:44:48.176349 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.176270 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-kube-api-access-8fw2x" (OuterVolumeSpecName: "kube-api-access-8fw2x") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "kube-api-access-8fw2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:44:48.176349 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.176313 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:44:48.176418 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.176404 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b8ca720d-6a5f-4cd8-9bd3-307ea417a206" (UID: "b8ca720d-6a5f-4cd8-9bd3-307ea417a206"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 00:44:48.193720 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.193695 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:48.209683 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.209662 2551 generic.go:356] "Generic (PLEG): container finished" podID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerID="9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375" exitCode=0 Aug 04 00:44:48.209789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.209708 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" Aug 04 00:44:48.209789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.209751 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" event={"ID":"b8ca720d-6a5f-4cd8-9bd3-307ea417a206","Type":"ContainerDied","Data":"9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375"} Aug 04 00:44:48.209936 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.209829 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m" event={"ID":"b8ca720d-6a5f-4cd8-9bd3-307ea417a206","Type":"ContainerDied","Data":"459eafe4d8de572bed19eae96499863359f8d09cc3f7c323366673cfd8dc04c8"} Aug 04 00:44:48.209936 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.209865 2551 scope.go:117] "RemoveContainer" containerID="9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375" Aug 04 00:44:48.219356 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.219342 2551 scope.go:117] "RemoveContainer" containerID="638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55" Aug 04 00:44:48.227664 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.227647 2551 scope.go:117] "RemoveContainer" containerID="9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375" Aug 04 00:44:48.227943 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:44:48.227926 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375\": container with ID starting with 9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375 not found: ID does not exist" containerID="9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375" Aug 04 00:44:48.227987 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.227952 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375"} err="failed to get container status \"9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375\": rpc error: code = NotFound desc = could not find container \"9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375\": container with ID starting with 9e830922ecf8678b2489d0578d827b622ffdef771fdf75c07b03eb60355c6375 not found: ID does not exist" Aug 04 00:44:48.227987 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.227968 2551 scope.go:117] "RemoveContainer" containerID="638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55" Aug 04 00:44:48.228181 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:44:48.228163 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55\": container with ID starting with 638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55 not found: ID does not exist" containerID="638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55" Aug 04 00:44:48.228223 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.228186 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55"} err="failed to get container status \"638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55\": rpc error: code = NotFound desc = could not find container \"638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55\": container with ID starting with 638a1fb60f5b388750df3eacac4443e152fc94572c2460893936e298a8e32f55 not found: ID does not exist" Aug 04 00:44:48.231722 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.231692 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m"] Aug 04 00:44:48.235352 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.235331 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5d4def-openshift-default-69b6889864-mnj7m"] Aug 04 00:44:48.275527 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.275503 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fw2x\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-kube-api-access-8fw2x\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.275527 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.275525 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-podinfo\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.275645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.275535 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-workload-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.275645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.275545 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istiod-ca-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.275645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.275554 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-envoy\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.275645 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.275563 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8ca720d-6a5f-4cd8-9bd3-307ea417a206-istio-token\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:44:48.550185 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.550103 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:48.658783 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.658723 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:44:48.659577 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:48.659558 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" path="/var/lib/kubelet/pods/b8ca720d-6a5f-4cd8-9bd3-307ea417a206/volumes" Aug 04 00:44:50.521382 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:44:50.521345 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:45:03.973918 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:03.973888 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:11.723209 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:11.723165 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:12.180272 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:12.180236 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:12.470305 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:12.470226 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:12.570979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:12.570947 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:15.145682 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:15.145639 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:15.241186 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:15.241109 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:16.330146 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:16.330109 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:16.429781 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:16.429739 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:16.811259 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:16.811224 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:16.899417 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:16.899389 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:19.131418 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:19.131382 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:45:40.012389 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:40.012354 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:41.109118 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:41.109084 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:52.745571 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:52.745533 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:52.795237 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:52.795202 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:58.249743 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:58.249707 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:58.324505 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:58.324475 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:58.382033 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:58.382001 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:58.446671 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:58.446639 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:45:58.802335 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:45:58.802304 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:00.632897 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:00.632853 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:46:04.532186 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:04.532153 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:04.589123 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:04.589085 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:06.230581 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:06.230543 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:20.766080 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:20.766043 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:20.834482 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:20.834450 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:20.907899 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:20.907868 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:20.967435 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:20.967399 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:21.308710 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:21.308678 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:22.816660 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:22.816626 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:46:26.972612 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:26.972574 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:27.029091 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:27.029056 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:27.236693 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:27.236573 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:27.447973 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:27.447942 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:27.771308 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:27.771276 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:28.040557 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.040483 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg"] Aug 04 00:46:28.040994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.040971 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" Aug 04 00:46:28.040994 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.040992 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" Aug 04 00:46:28.041170 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.041017 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" Aug 04 00:46:28.041170 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.041026 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" Aug 04 00:46:28.041170 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.041131 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" Aug 04 00:46:28.044456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.044437 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.046900 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.046882 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a5b2e0-openshift-default-dockercfg-75fwh\"" Aug 04 00:46:28.056826 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.056785 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg"] Aug 04 00:46:28.069952 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.069921 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:28.170015 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.169978 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-podinfo\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170015 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170021 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kfhn\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-kube-api-access-4kfhn\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170053 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istiod-ca-cert\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170106 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-token\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170131 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-certs\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170156 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-socket\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170174 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-envoy\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170191 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-data\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.170269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.170216 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-credential-socket\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271293 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271253 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-podinfo\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271490 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271298 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kfhn\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-kube-api-access-4kfhn\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271490 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271335 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istiod-ca-cert\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271490 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271379 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-token\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271490 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271405 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-certs\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271535 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-socket\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271579 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-envoy\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271607 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-data\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271636 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-credential-socket\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271828 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-certs\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.271975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.271884 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-socket\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.272104 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.272062 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-credential-socket\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.272183 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.272162 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-data\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.272463 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.272442 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istiod-ca-cert\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.274032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.274011 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-envoy\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.274548 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.274528 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-podinfo\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.280032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.280005 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kfhn\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-kube-api-access-4kfhn\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.280122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.280107 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-token\") pod \"e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.358237 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.358159 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:28.505747 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.505720 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg"] Aug 04 00:46:28.507699 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:46:28.507673 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62deb26d_9ae2_4f5b_be38_3a6d6066ae10.slice/crio-6cf3b0cd333e5a6ff481c44d9590e96c0ba2d5575819c833e10b1b436d587484 WatchSource:0}: Error finding container 6cf3b0cd333e5a6ff481c44d9590e96c0ba2d5575819c833e10b1b436d587484: Status 404 returned error can't find the container with id 6cf3b0cd333e5a6ff481c44d9590e96c0ba2d5575819c833e10b1b436d587484 Aug 04 00:46:28.510389 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.510350 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:46:28.510475 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.510438 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:46:28.510514 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.510480 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:46:28.564870 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.564839 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:28.748070 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.748033 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" event={"ID":"62deb26d-9ae2-4f5b-be38-3a6d6066ae10","Type":"ContainerStarted","Data":"6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583"} Aug 04 00:46:28.748269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.748097 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" event={"ID":"62deb26d-9ae2-4f5b-be38-3a6d6066ae10","Type":"ContainerStarted","Data":"6cf3b0cd333e5a6ff481c44d9590e96c0ba2d5575819c833e10b1b436d587484"} Aug 04 00:46:28.785472 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:28.785418 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podStartSLOduration=0.785404714 podStartE2EDuration="785.404714ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:46:28.781655819 +0000 UTC m=+2654.662103382" watchObservedRunningTime="2026-08-04 00:46:28.785404714 +0000 UTC m=+2654.665852253" Aug 04 00:46:29.358549 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:29.358519 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:29.363383 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:29.363356 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:29.753743 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:29.753714 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:29.754704 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:29.754685 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:30.694875 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:30.694840 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:30.790760 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:30.790729 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:31.806219 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:31.806180 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:32.007855 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:32.007815 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:32.304734 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:32.304699 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:32.402236 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:32.402206 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:34.122765 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:34.122731 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:46:43.845078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:43.844979 2551 generic.go:356] "Generic (PLEG): container finished" podID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerID="6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583" exitCode=0 Aug 04 00:46:43.845078 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:43.845054 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" event={"ID":"62deb26d-9ae2-4f5b-be38-3a6d6066ae10","Type":"ContainerDied","Data":"6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583"} Aug 04 00:46:43.845509 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:43.845494 2551 scope.go:117] "RemoveContainer" containerID="6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583" Aug 04 00:46:43.846662 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:43.846632 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:46:43.846744 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:43.846712 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:46:43.846789 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:43.846757 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:46:44.753921 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:44.753890 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:44.853187 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:44.853157 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" event={"ID":"62deb26d-9ae2-4f5b-be38-3a6d6066ae10","Type":"ContainerStarted","Data":"4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c"} Aug 04 00:46:45.358567 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:45.358531 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:45.359779 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:45.359753 2551 patch_prober.go:28] interesting pod/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Aug 04 00:46:45.359919 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:45.359824 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Aug 04 00:46:46.359474 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:46.359436 2551 patch_prober.go:28] interesting pod/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Aug 04 00:46:46.359874 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:46.359491 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Aug 04 00:46:47.358787 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:47.358747 2551 patch_prober.go:28] interesting pod/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Aug 04 00:46:47.358981 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:47.358832 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Aug 04 00:46:48.359120 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:48.359081 2551 patch_prober.go:28] interesting pod/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Aug 04 00:46:48.359578 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:48.359140 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Aug 04 00:46:49.359417 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:49.359378 2551 patch_prober.go:28] interesting pod/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Aug 04 00:46:49.359824 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:49.359435 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Aug 04 00:46:50.362719 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:50.362686 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:50.363142 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:50.363068 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:50.363601 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:50.363586 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 00:46:55.611922 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:55.611885 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:55.925602 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:55.925571 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:56.250948 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:56.250867 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:56.413462 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:56.413425 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:59.538316 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:59.538281 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:46:59.632482 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:46:59.632449 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:47:00.732632 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:47:00.732599 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:47:00.879731 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:47:00.879684 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:47:01.217403 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:47:01.217353 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:47:01.318988 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:47:01.318948 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:47:03.329515 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:47:03.329480 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:47:24.466699 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:47:24.466620 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:48:06.497985 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:48:06.497948 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:48:07.830955 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:48:07.830918 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:48:53.837508 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:48:53.837467 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:48:56.223770 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:48:56.223694 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:49:04.441122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:49:04.441087 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:49:20.925451 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:49:20.925417 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:49:24.344629 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:49:24.344595 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:49:36.330391 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:49:36.330352 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:49:48.019772 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:49:48.019735 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:50:02.827345 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:50:02.827312 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:50:14.429369 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:50:14.429290 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:50:29.330455 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:50:29.330415 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:50:41.132437 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:50:41.132402 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:50:55.632656 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:50:55.632620 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:51:06.829956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:06.829916 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:51:08.117892 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:08.117853 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:51:22.121351 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:22.121312 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:51:33.720408 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:33.720372 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:51:48.733653 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:48.733614 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:51:52.524112 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:52.524081 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:51:53.416669 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:53.416630 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:51:58.925596 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:51:58.925556 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:52:14.877934 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:52:14.877900 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:52:16.933107 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:52:16.933075 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:52:38.855356 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:52:38.855316 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:53:08.744979 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:53:08.744908 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:53:11.529808 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:53:11.529760 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:53:46.624134 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:53:46.624103 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:53:58.136234 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:53:58.136202 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:54:10.732497 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:10.732465 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:54:13.368244 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:13.368213 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:54:14.539011 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:14.538981 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:54:15.448604 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:15.448565 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:54:20.722750 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:20.722705 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:54:29.135733 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:29.135668 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:54:31.524632 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:31.524596 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:54:59.731281 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:54:59.731252 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:55:01.522561 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:55:01.522526 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:55:42.623263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:55:42.623231 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:55:50.919978 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:55:50.919932 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:56:06.819618 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:06.819575 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:56:12.501543 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:12.501509 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:14.111202 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.111168 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:14.282975 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.282934 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7369-140579950876848"] Aug 04 00:56:14.283414 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.283400 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ca720d-6a5f-4cd8-9bd3-307ea417a206" containerName="istio-proxy" Aug 04 00:56:14.286485 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.286468 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:14.288906 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.288888 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s7lgt\"" Aug 04 00:56:14.293551 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.293514 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950876848"] Aug 04 00:56:14.398188 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.398156 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2plzp\" (UniqueName: \"kubernetes.io/projected/d794220c-c009-493f-9716-73added0be7b-kube-api-access-2plzp\") pod \"test-curl-7369-140579950876848\" (UID: \"d794220c-c009-493f-9716-73added0be7b\") " pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:14.499610 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.499569 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2plzp\" (UniqueName: \"kubernetes.io/projected/d794220c-c009-493f-9716-73added0be7b-kube-api-access-2plzp\") pod \"test-curl-7369-140579950876848\" (UID: \"d794220c-c009-493f-9716-73added0be7b\") " pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:14.508479 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.508454 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2plzp\" (UniqueName: \"kubernetes.io/projected/d794220c-c009-493f-9716-73added0be7b-kube-api-access-2plzp\") pod \"test-curl-7369-140579950876848\" (UID: \"d794220c-c009-493f-9716-73added0be7b\") " pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:14.600518 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.600490 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s7lgt\"" Aug 04 00:56:14.608488 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.608469 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:14.742449 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.742418 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950876848"] Aug 04 00:56:14.744336 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:56:14.744307 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd794220c_c009_493f_9716_73added0be7b.slice/crio-47c546c656f1817f40b539f0c426f8ada51f4bfa2d93cab2739c9b19a18aa431 WatchSource:0}: Error finding container 47c546c656f1817f40b539f0c426f8ada51f4bfa2d93cab2739c9b19a18aa431: Status 404 returned error can't find the container with id 47c546c656f1817f40b539f0c426f8ada51f4bfa2d93cab2739c9b19a18aa431 Aug 04 00:56:14.745598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.745582 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 00:56:14.829678 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:14.829646 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:56:15.008562 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:15.008481 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950876848" event={"ID":"d794220c-c009-493f-9716-73added0be7b","Type":"ContainerStarted","Data":"47c546c656f1817f40b539f0c426f8ada51f4bfa2d93cab2739c9b19a18aa431"} Aug 04 00:56:16.015957 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:16.015915 2551 generic.go:356] "Generic (PLEG): container finished" podID="d794220c-c009-493f-9716-73added0be7b" containerID="51c573fd164ac0b7b0c2887b99534b63568636f99a8ee48c6ed713bb34b0a35c" exitCode=0 Aug 04 00:56:16.016358 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:16.016008 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950876848" event={"ID":"d794220c-c009-493f-9716-73added0be7b","Type":"ContainerDied","Data":"51c573fd164ac0b7b0c2887b99534b63568636f99a8ee48c6ed713bb34b0a35c"} Aug 04 00:56:17.153200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.153176 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:17.175554 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.175451 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7369-140579950876848_d794220c-c009-493f-9716-73added0be7b/test-curl-7369-140579950876848/0.log" Aug 04 00:56:17.195694 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.195663 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7369-140579950876848"] Aug 04 00:56:17.201711 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.201684 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7369-140579950876848"] Aug 04 00:56:17.317697 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.317612 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7369-140579950883504"] Aug 04 00:56:17.318035 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.318021 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d794220c-c009-493f-9716-73added0be7b" containerName="test-curl-7369-140579950876848" Aug 04 00:56:17.318087 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.318038 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d794220c-c009-493f-9716-73added0be7b" containerName="test-curl-7369-140579950876848" Aug 04 00:56:17.318126 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.318110 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d794220c-c009-493f-9716-73added0be7b" containerName="test-curl-7369-140579950876848" Aug 04 00:56:17.321138 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.321121 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950883504" Aug 04 00:56:17.324437 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.324411 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2plzp\" (UniqueName: \"kubernetes.io/projected/d794220c-c009-493f-9716-73added0be7b-kube-api-access-2plzp\") pod \"d794220c-c009-493f-9716-73added0be7b\" (UID: \"d794220c-c009-493f-9716-73added0be7b\") " Aug 04 00:56:17.326998 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.326969 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d794220c-c009-493f-9716-73added0be7b-kube-api-access-2plzp" (OuterVolumeSpecName: "kube-api-access-2plzp") pod "d794220c-c009-493f-9716-73added0be7b" (UID: "d794220c-c009-493f-9716-73added0be7b"). InnerVolumeSpecName "kube-api-access-2plzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:56:17.327263 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.327245 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950883504"] Aug 04 00:56:17.425594 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.425552 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4fkb\" (UniqueName: \"kubernetes.io/projected/3305cda7-048d-4820-9706-789124ebea8f-kube-api-access-s4fkb\") pod \"test-curl-7369-140579950883504\" (UID: \"3305cda7-048d-4820-9706-789124ebea8f\") " pod="opendatahub/test-curl-7369-140579950883504" Aug 04 00:56:17.425771 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.425671 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2plzp\" (UniqueName: \"kubernetes.io/projected/d794220c-c009-493f-9716-73added0be7b-kube-api-access-2plzp\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:56:17.526188 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.526146 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4fkb\" (UniqueName: \"kubernetes.io/projected/3305cda7-048d-4820-9706-789124ebea8f-kube-api-access-s4fkb\") pod \"test-curl-7369-140579950883504\" (UID: \"3305cda7-048d-4820-9706-789124ebea8f\") " pod="opendatahub/test-curl-7369-140579950883504" Aug 04 00:56:17.534358 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.534331 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4fkb\" (UniqueName: \"kubernetes.io/projected/3305cda7-048d-4820-9706-789124ebea8f-kube-api-access-s4fkb\") pod \"test-curl-7369-140579950883504\" (UID: \"3305cda7-048d-4820-9706-789124ebea8f\") " pod="opendatahub/test-curl-7369-140579950883504" Aug 04 00:56:17.640357 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.640318 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950883504" Aug 04 00:56:17.775948 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:17.775920 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950883504"] Aug 04 00:56:17.777532 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:56:17.777503 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3305cda7_048d_4820_9706_789124ebea8f.slice/crio-621e15a7a054804428e62c4c09254c110606a5c3efdb255e68a7f9434a74ecba WatchSource:0}: Error finding container 621e15a7a054804428e62c4c09254c110606a5c3efdb255e68a7f9434a74ecba: Status 404 returned error can't find the container with id 621e15a7a054804428e62c4c09254c110606a5c3efdb255e68a7f9434a74ecba Aug 04 00:56:18.027765 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:18.027663 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950883504" event={"ID":"3305cda7-048d-4820-9706-789124ebea8f","Type":"ContainerStarted","Data":"621e15a7a054804428e62c4c09254c110606a5c3efdb255e68a7f9434a74ecba"} Aug 04 00:56:18.028999 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:18.028977 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47c546c656f1817f40b539f0c426f8ada51f4bfa2d93cab2739c9b19a18aa431" Aug 04 00:56:18.029115 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:18.029009 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:18.041064 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:56:18.041036 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7369-140579950876848\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="d794220c-c009-493f-9716-73added0be7b" pod="opendatahub/test-curl-7369-140579950876848" Aug 04 00:56:18.658987 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:18.658950 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d794220c-c009-493f-9716-73added0be7b" path="/var/lib/kubelet/pods/d794220c-c009-493f-9716-73added0be7b/volumes" Aug 04 00:56:19.035889 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:19.035780 2551 generic.go:356] "Generic (PLEG): container finished" podID="3305cda7-048d-4820-9706-789124ebea8f" containerID="e629c92422790fa413dcbb9ea94d155dc6a341cabf4f43025813393f357ce5f1" exitCode=0 Aug 04 00:56:19.035889 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:19.035872 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950883504" event={"ID":"3305cda7-048d-4820-9706-789124ebea8f","Type":"ContainerDied","Data":"e629c92422790fa413dcbb9ea94d155dc6a341cabf4f43025813393f357ce5f1"} Aug 04 00:56:20.179305 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.179280 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950883504" Aug 04 00:56:20.200483 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.200454 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7369-140579950883504_3305cda7-048d-4820-9706-789124ebea8f/test-curl-7369-140579950883504/0.log" Aug 04 00:56:20.220862 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.220745 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7369-140579950883504"] Aug 04 00:56:20.223721 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.223692 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7369-140579950883504"] Aug 04 00:56:20.353167 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.353071 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4fkb\" (UniqueName: \"kubernetes.io/projected/3305cda7-048d-4820-9706-789124ebea8f-kube-api-access-s4fkb\") pod \"3305cda7-048d-4820-9706-789124ebea8f\" (UID: \"3305cda7-048d-4820-9706-789124ebea8f\") " Aug 04 00:56:20.355131 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.355102 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3305cda7-048d-4820-9706-789124ebea8f-kube-api-access-s4fkb" (OuterVolumeSpecName: "kube-api-access-s4fkb") pod "3305cda7-048d-4820-9706-789124ebea8f" (UID: "3305cda7-048d-4820-9706-789124ebea8f"). InnerVolumeSpecName "kube-api-access-s4fkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:56:20.380288 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.380258 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:20.380671 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.380658 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3305cda7-048d-4820-9706-789124ebea8f" containerName="test-curl-7369-140579950883504" Aug 04 00:56:20.380720 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.380674 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3305cda7-048d-4820-9706-789124ebea8f" containerName="test-curl-7369-140579950883504" Aug 04 00:56:20.380769 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.380759 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3305cda7-048d-4820-9706-789124ebea8f" containerName="test-curl-7369-140579950883504" Aug 04 00:56:20.383766 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.383751 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:20.389702 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.389677 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:20.454268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.454234 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4fkb\" (UniqueName: \"kubernetes.io/projected/3305cda7-048d-4820-9706-789124ebea8f-kube-api-access-s4fkb\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:56:20.554880 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.554842 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ct5d\" (UniqueName: \"kubernetes.io/projected/d7878b47-4d32-4484-8282-0e4632c88160-kube-api-access-8ct5d\") pod \"test-curl-7369-140579950775584\" (UID: \"d7878b47-4d32-4484-8282-0e4632c88160\") " pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:20.655570 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.655541 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ct5d\" (UniqueName: \"kubernetes.io/projected/d7878b47-4d32-4484-8282-0e4632c88160-kube-api-access-8ct5d\") pod \"test-curl-7369-140579950775584\" (UID: \"d7878b47-4d32-4484-8282-0e4632c88160\") " pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:20.659539 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.659513 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3305cda7-048d-4820-9706-789124ebea8f" path="/var/lib/kubelet/pods/3305cda7-048d-4820-9706-789124ebea8f/volumes" Aug 04 00:56:20.663582 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.663559 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ct5d\" (UniqueName: \"kubernetes.io/projected/d7878b47-4d32-4484-8282-0e4632c88160-kube-api-access-8ct5d\") pod \"test-curl-7369-140579950775584\" (UID: \"d7878b47-4d32-4484-8282-0e4632c88160\") " pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:20.694524 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.694490 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:20.830761 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:20.830725 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:20.831845 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:56:20.831826 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7878b47_4d32_4484_8282_0e4632c88160.slice/crio-fcb5cbfc68622f828e11ed7261e3fc4bd78ea49188168cb41ddebc41635806cb WatchSource:0}: Error finding container fcb5cbfc68622f828e11ed7261e3fc4bd78ea49188168cb41ddebc41635806cb: Status 404 returned error can't find the container with id fcb5cbfc68622f828e11ed7261e3fc4bd78ea49188168cb41ddebc41635806cb Aug 04 00:56:21.048956 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:21.048864 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950775584" event={"ID":"d7878b47-4d32-4484-8282-0e4632c88160","Type":"ContainerStarted","Data":"fcb5cbfc68622f828e11ed7261e3fc4bd78ea49188168cb41ddebc41635806cb"} Aug 04 00:56:21.050156 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:21.050136 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950883504" Aug 04 00:56:21.050277 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:21.050135 2551 scope.go:117] "RemoveContainer" containerID="e629c92422790fa413dcbb9ea94d155dc6a341cabf4f43025813393f357ce5f1" Aug 04 00:56:22.057876 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:22.057836 2551 generic.go:356] "Generic (PLEG): container finished" podID="d7878b47-4d32-4484-8282-0e4632c88160" containerID="849a3aa518b5f694d6203ec6722cc99e032959d2b2f2e56d9fbc07db079f33e4" exitCode=0 Aug 04 00:56:22.058268 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:22.057920 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950775584" event={"ID":"d7878b47-4d32-4484-8282-0e4632c88160","Type":"ContainerDied","Data":"849a3aa518b5f694d6203ec6722cc99e032959d2b2f2e56d9fbc07db079f33e4"} Aug 04 00:56:22.331868 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:22.331753 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:56:23.195325 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.195300 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:23.214709 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.214616 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7369-140579950775584_d7878b47-4d32-4484-8282-0e4632c88160/test-curl-7369-140579950775584/0.log" Aug 04 00:56:23.239335 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.239311 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:23.243105 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.243081 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:23.358679 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.358593 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:23.359020 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.359006 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7878b47-4d32-4484-8282-0e4632c88160" containerName="test-curl-7369-140579950775584" Aug 04 00:56:23.359067 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.359022 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7878b47-4d32-4484-8282-0e4632c88160" containerName="test-curl-7369-140579950775584" Aug 04 00:56:23.359103 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.359086 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7878b47-4d32-4484-8282-0e4632c88160" containerName="test-curl-7369-140579950775584" Aug 04 00:56:23.362020 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.362001 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:23.368681 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.368611 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:23.379812 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.379770 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ct5d\" (UniqueName: \"kubernetes.io/projected/d7878b47-4d32-4484-8282-0e4632c88160-kube-api-access-8ct5d\") pod \"d7878b47-4d32-4484-8282-0e4632c88160\" (UID: \"d7878b47-4d32-4484-8282-0e4632c88160\") " Aug 04 00:56:23.381782 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.381756 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7878b47-4d32-4484-8282-0e4632c88160-kube-api-access-8ct5d" (OuterVolumeSpecName: "kube-api-access-8ct5d") pod "d7878b47-4d32-4484-8282-0e4632c88160" (UID: "d7878b47-4d32-4484-8282-0e4632c88160"). InnerVolumeSpecName "kube-api-access-8ct5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:56:23.481267 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.481233 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6486\" (UniqueName: \"kubernetes.io/projected/bb557431-c062-443b-be19-ee196f7e7151-kube-api-access-h6486\") pod \"test-curl-7369-140579950775584\" (UID: \"bb557431-c062-443b-be19-ee196f7e7151\") " pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:23.481422 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.481351 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ct5d\" (UniqueName: \"kubernetes.io/projected/d7878b47-4d32-4484-8282-0e4632c88160-kube-api-access-8ct5d\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:56:23.582499 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.582462 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6486\" (UniqueName: \"kubernetes.io/projected/bb557431-c062-443b-be19-ee196f7e7151-kube-api-access-h6486\") pod \"test-curl-7369-140579950775584\" (UID: \"bb557431-c062-443b-be19-ee196f7e7151\") " pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:23.591374 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.591341 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6486\" (UniqueName: \"kubernetes.io/projected/bb557431-c062-443b-be19-ee196f7e7151-kube-api-access-h6486\") pod \"test-curl-7369-140579950775584\" (UID: \"bb557431-c062-443b-be19-ee196f7e7151\") " pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:23.673788 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.673754 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:23.807920 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:23.807896 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:23.809873 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:56:23.809847 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb557431_c062_443b_be19_ee196f7e7151.slice/crio-ca259455b8a77dcfe104b7ba522525ef7a0a79f7455f38a65b21db9cc8b40c9f WatchSource:0}: Error finding container ca259455b8a77dcfe104b7ba522525ef7a0a79f7455f38a65b21db9cc8b40c9f: Status 404 returned error can't find the container with id ca259455b8a77dcfe104b7ba522525ef7a0a79f7455f38a65b21db9cc8b40c9f Aug 04 00:56:24.069843 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:24.069722 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950775584" event={"ID":"bb557431-c062-443b-be19-ee196f7e7151","Type":"ContainerStarted","Data":"ca259455b8a77dcfe104b7ba522525ef7a0a79f7455f38a65b21db9cc8b40c9f"} Aug 04 00:56:24.071076 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:24.071046 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:24.071185 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:24.071055 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcb5cbfc68622f828e11ed7261e3fc4bd78ea49188168cb41ddebc41635806cb" Aug 04 00:56:24.082648 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:24.082617 2551 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7369-140579950775584" oldPodUID="d7878b47-4d32-4484-8282-0e4632c88160" podUID="bb557431-c062-443b-be19-ee196f7e7151" Aug 04 00:56:24.659516 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:24.659485 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7878b47-4d32-4484-8282-0e4632c88160" path="/var/lib/kubelet/pods/d7878b47-4d32-4484-8282-0e4632c88160/volumes" Aug 04 00:56:25.078399 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:25.078299 2551 generic.go:356] "Generic (PLEG): container finished" podID="bb557431-c062-443b-be19-ee196f7e7151" containerID="5082c7162749bb7251498e67d76e5cba2ae07f37ed80832cfb849fc5193f9b2f" exitCode=0 Aug 04 00:56:25.078399 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:25.078355 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950775584" event={"ID":"bb557431-c062-443b-be19-ee196f7e7151","Type":"ContainerDied","Data":"5082c7162749bb7251498e67d76e5cba2ae07f37ed80832cfb849fc5193f9b2f"} Aug 04 00:56:26.230598 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.230569 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:26.250466 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.250432 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7369-140579950775584_bb557431-c062-443b-be19-ee196f7e7151/test-curl-7369-140579950775584/0.log" Aug 04 00:56:26.270908 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.270875 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:26.273041 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.273021 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7369-140579950775584"] Aug 04 00:56:26.393510 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.393478 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7369-140579950801232"] Aug 04 00:56:26.393945 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.393930 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb557431-c062-443b-be19-ee196f7e7151" containerName="test-curl-7369-140579950775584" Aug 04 00:56:26.393995 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.393949 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb557431-c062-443b-be19-ee196f7e7151" containerName="test-curl-7369-140579950775584" Aug 04 00:56:26.394038 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.394026 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb557431-c062-443b-be19-ee196f7e7151" containerName="test-curl-7369-140579950775584" Aug 04 00:56:26.397046 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.397030 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:26.402672 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.402648 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950801232"] Aug 04 00:56:26.409283 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.409261 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6486\" (UniqueName: \"kubernetes.io/projected/bb557431-c062-443b-be19-ee196f7e7151-kube-api-access-h6486\") pod \"bb557431-c062-443b-be19-ee196f7e7151\" (UID: \"bb557431-c062-443b-be19-ee196f7e7151\") " Aug 04 00:56:26.411358 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.411331 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb557431-c062-443b-be19-ee196f7e7151-kube-api-access-h6486" (OuterVolumeSpecName: "kube-api-access-h6486") pod "bb557431-c062-443b-be19-ee196f7e7151" (UID: "bb557431-c062-443b-be19-ee196f7e7151"). InnerVolumeSpecName "kube-api-access-h6486". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:56:26.510224 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.510189 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqmql\" (UniqueName: \"kubernetes.io/projected/cb8b418c-2bc2-416b-b3fb-6c0c20c842ba-kube-api-access-gqmql\") pod \"test-curl-7369-140579950801232\" (UID: \"cb8b418c-2bc2-416b-b3fb-6c0c20c842ba\") " pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:26.510395 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.510323 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6486\" (UniqueName: \"kubernetes.io/projected/bb557431-c062-443b-be19-ee196f7e7151-kube-api-access-h6486\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:56:26.611572 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.611538 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqmql\" (UniqueName: \"kubernetes.io/projected/cb8b418c-2bc2-416b-b3fb-6c0c20c842ba-kube-api-access-gqmql\") pod \"test-curl-7369-140579950801232\" (UID: \"cb8b418c-2bc2-416b-b3fb-6c0c20c842ba\") " pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:26.619822 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.619769 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqmql\" (UniqueName: \"kubernetes.io/projected/cb8b418c-2bc2-416b-b3fb-6c0c20c842ba-kube-api-access-gqmql\") pod \"test-curl-7369-140579950801232\" (UID: \"cb8b418c-2bc2-416b-b3fb-6c0c20c842ba\") " pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:26.659342 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.659265 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb557431-c062-443b-be19-ee196f7e7151" path="/var/lib/kubelet/pods/bb557431-c062-443b-be19-ee196f7e7151/volumes" Aug 04 00:56:26.708009 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.707978 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:26.840206 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:26.840183 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7369-140579950801232"] Aug 04 00:56:26.842818 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:56:26.842773 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb8b418c_2bc2_416b_b3fb_6c0c20c842ba.slice/crio-451fd382a933ad3028530e9d60f756e9bfc38c27cce1b166fcfa88927b4275d2 WatchSource:0}: Error finding container 451fd382a933ad3028530e9d60f756e9bfc38c27cce1b166fcfa88927b4275d2: Status 404 returned error can't find the container with id 451fd382a933ad3028530e9d60f756e9bfc38c27cce1b166fcfa88927b4275d2 Aug 04 00:56:27.097791 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:27.097710 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950801232" event={"ID":"cb8b418c-2bc2-416b-b3fb-6c0c20c842ba","Type":"ContainerStarted","Data":"451fd382a933ad3028530e9d60f756e9bfc38c27cce1b166fcfa88927b4275d2"} Aug 04 00:56:27.098990 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:27.098966 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950775584" Aug 04 00:56:27.099102 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:27.099009 2551 scope.go:117] "RemoveContainer" containerID="5082c7162749bb7251498e67d76e5cba2ae07f37ed80832cfb849fc5193f9b2f" Aug 04 00:56:28.106072 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:28.106035 2551 generic.go:356] "Generic (PLEG): container finished" podID="cb8b418c-2bc2-416b-b3fb-6c0c20c842ba" containerID="7b2ebc473c366c70f20398973cf1a821e0aed79e10ee883a39d623797e868c06" exitCode=0 Aug 04 00:56:28.106544 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:28.106082 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7369-140579950801232" event={"ID":"cb8b418c-2bc2-416b-b3fb-6c0c20c842ba","Type":"ContainerDied","Data":"7b2ebc473c366c70f20398973cf1a821e0aed79e10ee883a39d623797e868c06"} Aug 04 00:56:29.254828 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:29.254786 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:29.274659 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:29.274626 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7369-140579950801232_cb8b418c-2bc2-416b-b3fb-6c0c20c842ba/test-curl-7369-140579950801232/0.log" Aug 04 00:56:29.295165 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:29.295109 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7369-140579950801232"] Aug 04 00:56:29.300497 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:29.300464 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7369-140579950801232"] Aug 04 00:56:29.336200 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:29.336160 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqmql\" (UniqueName: \"kubernetes.io/projected/cb8b418c-2bc2-416b-b3fb-6c0c20c842ba-kube-api-access-gqmql\") pod \"cb8b418c-2bc2-416b-b3fb-6c0c20c842ba\" (UID: \"cb8b418c-2bc2-416b-b3fb-6c0c20c842ba\") " Aug 04 00:56:29.338344 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:29.338317 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb8b418c-2bc2-416b-b3fb-6c0c20c842ba-kube-api-access-gqmql" (OuterVolumeSpecName: "kube-api-access-gqmql") pod "cb8b418c-2bc2-416b-b3fb-6c0c20c842ba" (UID: "cb8b418c-2bc2-416b-b3fb-6c0c20c842ba"). InnerVolumeSpecName "kube-api-access-gqmql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:56:29.437221 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:29.437189 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqmql\" (UniqueName: \"kubernetes.io/projected/cb8b418c-2bc2-416b-b3fb-6c0c20c842ba-kube-api-access-gqmql\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:56:30.120290 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:30.120262 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:30.120290 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:30.120293 2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="451fd382a933ad3028530e9d60f756e9bfc38c27cce1b166fcfa88927b4275d2" Aug 04 00:56:30.130624 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:56:30.130597 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7369-140579950801232\" is forbidden: User \"system:node:ip-10-0-138-239.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-239.ec2.internal' and this object" podUID="cb8b418c-2bc2-416b-b3fb-6c0c20c842ba" pod="opendatahub/test-curl-7369-140579950801232" Aug 04 00:56:30.659090 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:30.659054 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb8b418c-2bc2-416b-b3fb-6c0c20c842ba" path="/var/lib/kubelet/pods/cb8b418c-2bc2-416b-b3fb-6c0c20c842ba/volumes" Aug 04 00:56:33.232688 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:33.232656 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:33.288390 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:33.288357 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:35.644581 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:35.644546 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:35.744217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:35.744184 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:36.776331 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:36.776291 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:37.268145 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:37.268113 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:37.367514 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:37.367474 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:37.466952 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:37.466915 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:56:39.835891 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:56:39.835855 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:57:00.622958 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:00.622921 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:01.263499 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:01.263464 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:01.574575 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:01.574492 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:01.858934 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:01.858854 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:04.828726 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:04.828689 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:04.927101 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:04.927068 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:05.958464 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:05.958430 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:06.268381 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:06.268284 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:06.565718 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:06.565638 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:06.660817 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:06.660764 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:08.840542 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:08.840507 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:57:29.799185 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:29.799152 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:53.432313 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:53.432280 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:57:56.026018 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:57:56.025981 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:58:12.418161 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:12.418126 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:20.537958 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:20.537920 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:58:32.180634 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.180592 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:32.468897 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.468810 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp"] Aug 04 00:58:32.469419 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.469404 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb8b418c-2bc2-416b-b3fb-6c0c20c842ba" containerName="test-curl-7369-140579950801232" Aug 04 00:58:32.469468 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.469423 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb8b418c-2bc2-416b-b3fb-6c0c20c842ba" containerName="test-curl-7369-140579950801232" Aug 04 00:58:32.469526 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.469514 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb8b418c-2bc2-416b-b3fb-6c0c20c842ba" containerName="test-curl-7369-140579950801232" Aug 04 00:58:32.473036 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.473015 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.475882 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.475782 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-974a1b-openshift-default-dockercfg-8977s\"" Aug 04 00:58:32.484912 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.484883 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp"] Aug 04 00:58:32.496462 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.496427 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:32.595860 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.595823 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-token\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.595874 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-credential-socket\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.595958 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e83481ca-d176-4315-86e9-8f49027ed7fd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.596013 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-data\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596143 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.596113 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-socket\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596184 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.596173 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-envoy\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596220 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.596196 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-podinfo\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596257 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.596226 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-kube-api-access-sdhvc\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.596308 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.596289 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-certs\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697064 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697023 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-data\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697082 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-socket\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697145 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-envoy\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697177 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-podinfo\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697271 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697216 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-kube-api-access-sdhvc\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697500 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697263 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-certs\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697500 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697312 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-token\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697500 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697361 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-credential-socket\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697500 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697404 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e83481ca-d176-4315-86e9-8f49027ed7fd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697500 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697484 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-data\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697762 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697656 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-certs\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697837 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697785 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-socket\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.697898 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.697848 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-credential-socket\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.698134 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.698117 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e83481ca-d176-4315-86e9-8f49027ed7fd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.699705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.699667 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-envoy\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.699705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.699682 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-podinfo\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.705590 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.705563 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-token\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.706005 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.705989 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-kube-api-access-sdhvc\") pod \"e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.785427 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.785331 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:32.890407 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.890375 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:32.931819 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.931769 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp"] Aug 04 00:58:32.934559 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:58:32.934526 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83481ca_d176_4315_86e9_8f49027ed7fd.slice/crio-5d894fd355c66cb23a9febaf769027eff57a477063804ae8500391cdd6ba5c28 WatchSource:0}: Error finding container 5d894fd355c66cb23a9febaf769027eff57a477063804ae8500391cdd6ba5c28: Status 404 returned error can't find the container with id 5d894fd355c66cb23a9febaf769027eff57a477063804ae8500391cdd6ba5c28 Aug 04 00:58:32.936863 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.936825 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:58:32.936972 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.936910 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:58:32.936972 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:32.936953 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:58:33.812320 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:33.812283 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" event={"ID":"e83481ca-d176-4315-86e9-8f49027ed7fd","Type":"ContainerStarted","Data":"fe9f8d3c9df78ef48e46484f530f7dfb87123782b2c18d0f523b5ecdc7ddeca7"} Aug 04 00:58:33.812892 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:33.812333 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" event={"ID":"e83481ca-d176-4315-86e9-8f49027ed7fd","Type":"ContainerStarted","Data":"5d894fd355c66cb23a9febaf769027eff57a477063804ae8500391cdd6ba5c28"} Aug 04 00:58:33.833185 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:33.833113 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podStartSLOduration=1.833097023 podStartE2EDuration="1.833097023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:58:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:58:33.832344425 +0000 UTC m=+3379.712791964" watchObservedRunningTime="2026-08-04 00:58:33.833097023 +0000 UTC m=+3379.713544563" Aug 04 00:58:34.786132 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:34.786090 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:34.790967 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:34.790940 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:34.817721 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:34.817686 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:34.818677 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:34.818659 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:35.733119 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:35.733088 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:35.829083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:35.829047 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:36.830630 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.830596 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj"] Aug 04 00:58:36.834708 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.834684 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.837122 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.837099 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-974a1b-serving-cert\"" Aug 04 00:58:36.837756 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.837723 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.837942 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.837919 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pcrn\" (UniqueName: \"kubernetes.io/projected/f589d2cd-e4ae-493c-b671-0ec8e67ac817-kube-api-access-4pcrn\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.838141 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.838118 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589d2cd-e4ae-493c-b671-0ec8e67ac817-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.842381 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.842353 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj"] Aug 04 00:58:36.860379 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.860342 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:36.938899 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.938864 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pcrn\" (UniqueName: \"kubernetes.io/projected/f589d2cd-e4ae-493c-b671-0ec8e67ac817-kube-api-access-4pcrn\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.939114 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.938911 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589d2cd-e4ae-493c-b671-0ec8e67ac817-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.939114 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.938986 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.939114 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:58:36.939072 2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-serving-cert: secret "maas-api-e2e-ipp-cleanup-974a1b-serving-cert" not found Aug 04 00:58:36.939324 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:58:36.939131 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls podName:f589d2cd-e4ae-493c-b671-0ec8e67ac817 nodeName:}" failed. No retries permitted until 2026-08-04 00:58:37.439115888 +0000 UTC m=+3383.319563405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" (UID: "f589d2cd-e4ae-493c-b671-0ec8e67ac817") : secret "maas-api-e2e-ipp-cleanup-974a1b-serving-cert" not found Aug 04 00:58:36.939672 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.939648 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589d2cd-e4ae-493c-b671-0ec8e67ac817-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:36.947556 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:36.947527 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pcrn\" (UniqueName: \"kubernetes.io/projected/f589d2cd-e4ae-493c-b671-0ec8e67ac817-kube-api-access-4pcrn\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:37.336355 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.336322 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:37.442586 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.442532 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:37.445278 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.445244 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:37.449974 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.449949 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:37.479646 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.479614 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:37.579823 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.579761 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:58:37.611552 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.611521 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj"] Aug 04 00:58:37.613372 ip-10-0-138-239 kubenswrapper[2551]: W0804 00:58:37.613329 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf589d2cd_e4ae_493c_b671_0ec8e67ac817.slice/crio-6318e2e34aa00075a6ad3e018d6e56cbdff57deb2a45663f946a94b6f25e456b WatchSource:0}: Error finding container 6318e2e34aa00075a6ad3e018d6e56cbdff57deb2a45663f946a94b6f25e456b: Status 404 returned error can't find the container with id 6318e2e34aa00075a6ad3e018d6e56cbdff57deb2a45663f946a94b6f25e456b Aug 04 00:58:37.837117 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:37.837075 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" event={"ID":"f589d2cd-e4ae-493c-b671-0ec8e67ac817","Type":"ContainerStarted","Data":"6318e2e34aa00075a6ad3e018d6e56cbdff57deb2a45663f946a94b6f25e456b"} Aug 04 00:58:38.844733 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:38.844694 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" event={"ID":"f589d2cd-e4ae-493c-b671-0ec8e67ac817","Type":"ContainerStarted","Data":"d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3"} Aug 04 00:58:38.845164 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:38.844822 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:38.863292 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:38.863235 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" podStartSLOduration=2.504182726 podStartE2EDuration="2.863220848s" totalImagesPullingTime="359.038122ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 00:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 00:58:38.861346082 +0000 UTC m=+3384.741793643" watchObservedRunningTime="2026-08-04 00:58:38.863220848 +0000 UTC m=+3384.743668389" Aug 04 00:58:42.326921 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:42.326878 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:58:44.858225 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:44.858195 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:58:50.859032 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:50.858981 2551 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Aug 04 00:58:50.859444 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:50.859056 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Aug 04 00:58:52.938772 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:52.938738 2551 generic.go:356] "Generic (PLEG): container finished" podID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerID="fe9f8d3c9df78ef48e46484f530f7dfb87123782b2c18d0f523b5ecdc7ddeca7" exitCode=0 Aug 04 00:58:52.939217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:52.938830 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" event={"ID":"e83481ca-d176-4315-86e9-8f49027ed7fd","Type":"ContainerDied","Data":"fe9f8d3c9df78ef48e46484f530f7dfb87123782b2c18d0f523b5ecdc7ddeca7"} Aug 04 00:58:52.939217 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:52.939213 2551 scope.go:117] "RemoveContainer" containerID="fe9f8d3c9df78ef48e46484f530f7dfb87123782b2c18d0f523b5ecdc7ddeca7" Aug 04 00:58:52.940411 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:52.940384 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:58:52.940477 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:52.940455 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:58:52.940517 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:52.940484 2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 00:58:53.946309 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:53.946276 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" event={"ID":"e83481ca-d176-4315-86e9-8f49027ed7fd","Type":"ContainerStarted","Data":"6da44902fef54765a32049c4606559a8dcac03d8f2b777555dff0fa572a61a20"} Aug 04 00:58:54.786587 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:54.786541 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:54.786936 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:54.786651 2551 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Aug 04 00:58:54.787147 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:54.787110 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Aug 04 00:58:55.786821 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:55.786756 2551 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Aug 04 00:58:55.787424 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:55.787389 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Aug 04 00:58:56.786594 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:56.786557 2551 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Aug 04 00:58:56.786913 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:56.786882 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Aug 04 00:58:57.786573 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:57.786534 2551 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Aug 04 00:58:57.787323 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:57.786890 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Aug 04 00:58:58.786269 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:58.786230 2551 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Aug 04 00:58:58.786559 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:58.786530 2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Aug 04 00:58:59.789891 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:59.789859 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:59.790304 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:59.790123 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:58:59.790706 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:58:59.790687 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:59:00.684511 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:00.684475 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:07.543090 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.543006 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj"] Aug 04 00:59:07.543643 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.543220 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" podUID="f589d2cd-e4ae-493c-b671-0ec8e67ac817" containerName="maas-api" containerID="cri-o://d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3" gracePeriod=30 Aug 04 00:59:07.580925 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.580893 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:07.655821 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.655764 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:07.729754 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.729724 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:07.788718 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.788641 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:07.791244 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.791174 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:59:07.929062 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.929030 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls\") pod \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " Aug 04 00:59:07.929257 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.929138 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pcrn\" (UniqueName: \"kubernetes.io/projected/f589d2cd-e4ae-493c-b671-0ec8e67ac817-kube-api-access-4pcrn\") pod \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " Aug 04 00:59:07.929257 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.929177 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589d2cd-e4ae-493c-b671-0ec8e67ac817-openshift-service-ca\") pod \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\" (UID: \"f589d2cd-e4ae-493c-b671-0ec8e67ac817\") " Aug 04 00:59:07.929652 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.929619 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f589d2cd-e4ae-493c-b671-0ec8e67ac817-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f589d2cd-e4ae-493c-b671-0ec8e67ac817" (UID: "f589d2cd-e4ae-493c-b671-0ec8e67ac817"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:59:07.931414 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.931379 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f589d2cd-e4ae-493c-b671-0ec8e67ac817" (UID: "f589d2cd-e4ae-493c-b671-0ec8e67ac817"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 00:59:07.931535 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:07.931448 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f589d2cd-e4ae-493c-b671-0ec8e67ac817-kube-api-access-4pcrn" (OuterVolumeSpecName: "kube-api-access-4pcrn") pod "f589d2cd-e4ae-493c-b671-0ec8e67ac817" (UID: "f589d2cd-e4ae-493c-b671-0ec8e67ac817"). InnerVolumeSpecName "kube-api-access-4pcrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:59:08.030025 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.029981 2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f589d2cd-e4ae-493c-b671-0ec8e67ac817-maas-api-tls\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:08.030025 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.030027 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pcrn\" (UniqueName: \"kubernetes.io/projected/f589d2cd-e4ae-493c-b671-0ec8e67ac817-kube-api-access-4pcrn\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:08.030296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.030043 2551 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589d2cd-e4ae-493c-b671-0ec8e67ac817-openshift-service-ca\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:08.030296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.030136 2551 generic.go:356] "Generic (PLEG): container finished" podID="f589d2cd-e4ae-493c-b671-0ec8e67ac817" containerID="d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3" exitCode=0 Aug 04 00:59:08.030296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.030191 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" Aug 04 00:59:08.030296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.030221 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" event={"ID":"f589d2cd-e4ae-493c-b671-0ec8e67ac817","Type":"ContainerDied","Data":"d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3"} Aug 04 00:59:08.030296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.030274 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj" event={"ID":"f589d2cd-e4ae-493c-b671-0ec8e67ac817","Type":"ContainerDied","Data":"6318e2e34aa00075a6ad3e018d6e56cbdff57deb2a45663f946a94b6f25e456b"} Aug 04 00:59:08.030296 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.030300 2551 scope.go:117] "RemoveContainer" containerID="d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3" Aug 04 00:59:08.040867 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.040852 2551 scope.go:117] "RemoveContainer" containerID="d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3" Aug 04 00:59:08.041128 ip-10-0-138-239 kubenswrapper[2551]: E0804 00:59:08.041109 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3\": container with ID starting with d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3 not found: ID does not exist" containerID="d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3" Aug 04 00:59:08.041184 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.041136 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3"} err="failed to get container status \"d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3\": rpc error: code = NotFound desc = could not find container \"d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3\": container with ID starting with d75711ba07dc64855092c982e1267bb09795c16ab4e79d683633e2176a5afcc3 not found: ID does not exist" Aug 04 00:59:08.052700 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.052676 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj"] Aug 04 00:59:08.056649 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.056624 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-974a1b-6f7c55ddbb-dtvbj"] Aug 04 00:59:08.117658 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.117623 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:08.659875 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:08.659842 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f589d2cd-e4ae-493c-b671-0ec8e67ac817" path="/var/lib/kubelet/pods/f589d2cd-e4ae-493c-b671-0ec8e67ac817/volumes" Aug 04 00:59:10.520083 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:10.520041 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:59:13.702206 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:13.702173 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:13.773634 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:13.773599 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:13.946211 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:13.946180 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp"] Aug 04 00:59:13.946410 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:13.946386 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" containerID="cri-o://6da44902fef54765a32049c4606559a8dcac03d8f2b777555dff0fa572a61a20" gracePeriod=30 Aug 04 00:59:13.961402 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:13.961324 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:14.175051 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:14.175015 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:14.792517 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:14.792485 2551 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 00:59:14.792909 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:14.792550 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 00:59:19.095611 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.095572 2551 generic.go:356] "Generic (PLEG): container finished" podID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerID="6da44902fef54765a32049c4606559a8dcac03d8f2b777555dff0fa572a61a20" exitCode=0 Aug 04 00:59:19.096103 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.095746 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" event={"ID":"e83481ca-d176-4315-86e9-8f49027ed7fd","Type":"ContainerDied","Data":"6da44902fef54765a32049c4606559a8dcac03d8f2b777555dff0fa572a61a20"} Aug 04 00:59:19.096103 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.095834 2551 scope.go:117] "RemoveContainer" containerID="fe9f8d3c9df78ef48e46484f530f7dfb87123782b2c18d0f523b5ecdc7ddeca7" Aug 04 00:59:19.205959 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.205927 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:59:19.331222 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331133 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-envoy\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331222 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331187 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-kube-api-access-sdhvc\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331279 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-certs\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331316 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-socket\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331353 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-credential-socket\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331382 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-podinfo\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331456 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331408 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-token\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331461 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e83481ca-d176-4315-86e9-8f49027ed7fd-istiod-ca-cert\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331516 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:59:19.331705 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331544 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-data\") pod \"e83481ca-d176-4315-86e9-8f49027ed7fd\" (UID: \"e83481ca-d176-4315-86e9-8f49027ed7fd\") " Aug 04 00:59:19.331905 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331886 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-certs\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.331905 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331884 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:59:19.332024 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331915 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e83481ca-d176-4315-86e9-8f49027ed7fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 00:59:19.332024 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.331971 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:59:19.332133 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.332108 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:59:19.333531 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.333505 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:59:19.333615 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.333559 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-kube-api-access-sdhvc" (OuterVolumeSpecName: "kube-api-access-sdhvc") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "kube-api-access-sdhvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 00:59:19.333759 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.333738 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 00:59:19.333924 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.333906 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e83481ca-d176-4315-86e9-8f49027ed7fd" (UID: "e83481ca-d176-4315-86e9-8f49027ed7fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 00:59:19.432614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432575 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-workload-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.432614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432608 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-credential-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.432614 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432621 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-podinfo\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.432901 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432632 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-token\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.432901 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432644 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e83481ca-d176-4315-86e9-8f49027ed7fd-istiod-ca-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.432901 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432654 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-data\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.432901 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432664 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e83481ca-d176-4315-86e9-8f49027ed7fd-istio-envoy\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:19.432901 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:19.432674 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/e83481ca-d176-4315-86e9-8f49027ed7fd-kube-api-access-sdhvc\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 00:59:20.102525 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:20.102492 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" Aug 04 00:59:20.103046 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:20.102526 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp" event={"ID":"e83481ca-d176-4315-86e9-8f49027ed7fd","Type":"ContainerDied","Data":"5d894fd355c66cb23a9febaf769027eff57a477063804ae8500391cdd6ba5c28"} Aug 04 00:59:20.103046 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:20.102587 2551 scope.go:117] "RemoveContainer" containerID="6da44902fef54765a32049c4606559a8dcac03d8f2b777555dff0fa572a61a20" Aug 04 00:59:20.125368 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:20.125339 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp"] Aug 04 00:59:20.129173 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:20.129147 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-974a1b-openshift-default-7c85b974b4-wvhlp"] Aug 04 00:59:20.659292 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:20.659261 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" path="/var/lib/kubelet/pods/e83481ca-d176-4315-86e9-8f49027ed7fd/volumes" Aug 04 00:59:30.731315 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:30.731280 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 00:59:33.048754 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:33.048713 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:33.149195 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:33.149163 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:33.220577 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:33.220539 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:33.287065 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:33.287026 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:33.617363 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:33.617332 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:38.600685 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:38.600650 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 00:59:39.729612 ip-10-0-138-239 kubenswrapper[2551]: I0804 00:59:39.729575 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 01:00:06.233196 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:06.233159 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:15.207271 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:15.207237 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:15.593592 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:15.593562 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:21.071661 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:21.071620 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:26.924245 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:26.924211 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:27.235753 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:27.235662 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:27.314322 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:27.314292 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:27.432415 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:27.432371 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:38.176424 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:38.176381 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:38.273557 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:38.273527 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:38.374611 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:38.374582 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:38.431166 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:38.431083 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:38.767928 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:38.767835 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:40.641592 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:40.641554 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 01:00:44.759246 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:44.759207 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:44.859297 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:44.859261 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:45.039242 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:45.039147 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:45.240033 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:45.240002 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:55.826211 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:55.826181 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:55.923644 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:55.923602 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:56.004972 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:56.004936 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:56.064504 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:56.064469 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:56.374116 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:56.374081 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:00:58.025643 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:00:58.025603 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 01:01:02.487680 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:02.487651 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:02.555588 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:02.555556 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:02.727173 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:02.727081 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg"] Aug 04 01:01:02.727357 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:02.727328 2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" containerID="cri-o://4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c" gracePeriod=30 Aug 04 01:01:02.749887 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:02.749860 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:02.980400 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:02.980298 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:05.365661 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:05.365623 2551 patch_prober.go:28] interesting pod/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 01:01:05.366059 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:05.365683 2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 01:01:07.977153 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:07.977128 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 01:01:08.035372 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035297 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-envoy\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035372 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035346 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-socket\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035588 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035371 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-credential-socket\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035588 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035403 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-certs\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035588 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035446 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-podinfo\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035588 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035495 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-data\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035588 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035531 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-token\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035881 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035609 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kfhn\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-kube-api-access-4kfhn\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035881 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035634 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istiod-ca-cert\") pod \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\" (UID: \"62deb26d-9ae2-4f5b-be38-3a6d6066ae10\") " Aug 04 01:01:08.035881 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035666 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 01:01:08.035881 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035686 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 01:01:08.035881 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.035736 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 01:01:08.036143 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.036062 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 01:01:08.036143 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.036103 2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.036143 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.036123 2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-credential-socket\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.036143 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.036138 2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-workload-certs\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.036143 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.036136 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-data" (OuterVolumeSpecName: "istio-data") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 01:01:08.037585 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.037559 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 01:01:08.037762 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.037733 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-kube-api-access-4kfhn" (OuterVolumeSpecName: "kube-api-access-4kfhn") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "kube-api-access-4kfhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 01:01:08.037914 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.037809 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-token" (OuterVolumeSpecName: "istio-token") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 01:01:08.037914 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.037815 2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62deb26d-9ae2-4f5b-be38-3a6d6066ae10" (UID: "62deb26d-9ae2-4f5b-be38-3a6d6066ae10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 01:01:08.136628 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.136602 2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-podinfo\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.136628 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.136627 2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-data\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.136628 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.136635 2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-token\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.136835 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.136643 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kfhn\" (UniqueName: \"kubernetes.io/projected/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-kube-api-access-4kfhn\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.136835 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.136653 2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istiod-ca-cert\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.136835 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.136662 2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62deb26d-9ae2-4f5b-be38-3a6d6066ae10-istio-envoy\") on node \"ip-10-0-138-239.ec2.internal\" DevicePath \"\"" Aug 04 01:01:08.713834 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.713782 2551 generic.go:356] "Generic (PLEG): container finished" podID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerID="4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c" exitCode=0 Aug 04 01:01:08.714000 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.713828 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" event={"ID":"62deb26d-9ae2-4f5b-be38-3a6d6066ae10","Type":"ContainerDied","Data":"4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c"} Aug 04 01:01:08.714000 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.713859 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" Aug 04 01:01:08.714000 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.713894 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg" event={"ID":"62deb26d-9ae2-4f5b-be38-3a6d6066ae10","Type":"ContainerDied","Data":"6cf3b0cd333e5a6ff481c44d9590e96c0ba2d5575819c833e10b1b436d587484"} Aug 04 01:01:08.714000 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.713921 2551 scope.go:117] "RemoveContainer" containerID="4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c" Aug 04 01:01:08.723029 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.723010 2551 scope.go:117] "RemoveContainer" containerID="6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583" Aug 04 01:01:08.734334 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.734284 2551 scope.go:117] "RemoveContainer" containerID="4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c" Aug 04 01:01:08.734657 ip-10-0-138-239 kubenswrapper[2551]: E0804 01:01:08.734629 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c\": container with ID starting with 4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c not found: ID does not exist" containerID="4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c" Aug 04 01:01:08.734765 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.734682 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c"} err="failed to get container status \"4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c\": rpc error: code = NotFound desc = could not find container \"4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c\": container with ID starting with 4290e64d722c9343b639ec54e4f0064609ec99d8e2f5cc88e740b90de28ad71c not found: ID does not exist" Aug 04 01:01:08.734765 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.734707 2551 scope.go:117] "RemoveContainer" containerID="6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583" Aug 04 01:01:08.734765 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.734756 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg"] Aug 04 01:01:08.735014 ip-10-0-138-239 kubenswrapper[2551]: E0804 01:01:08.734997 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583\": container with ID starting with 6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583 not found: ID does not exist" containerID="6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583" Aug 04 01:01:08.735062 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.735020 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583"} err="failed to get container status \"6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583\": rpc error: code = NotFound desc = could not find container \"6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583\": container with ID starting with 6468d6d67b39b8da3ab5fcefcf6a2a4f9ee7579ff2a8cdf5dbb09ae1586ab583 not found: ID does not exist" Aug 04 01:01:08.737040 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:08.737022 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a5b2e0-openshift-default-768b78d4dd-wn6bg"] Aug 04 01:01:10.659221 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:10.659190 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" path="/var/lib/kubelet/pods/62deb26d-9ae2-4f5b-be38-3a6d6066ae10/volumes" Aug 04 01:01:13.591133 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:13.591102 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:13.661351 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:13.661319 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:13.727991 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:13.727956 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:13.787950 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:13.787915 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:14.137680 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:14.137643 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:15.726730 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:15.726689 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-qm2pq"] Aug 04 01:01:20.244863 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:20.244826 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:20.344495 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:20.344464 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:20.500048 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:20.499949 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:20.724257 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:20.724224 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:26.591381 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:26.591353 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-npdcp_2326363c-2143-4dec-8592-e5fdd7ab6dac/server/0.log" Aug 04 01:01:26.705216 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:26.705187 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2fvbv_bf451cea-1e4b-4100-9ebd-8dc323e00e92/manager/0.log" Aug 04 01:01:27.058854 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:27.058823 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8594b76987-wqh9c_0708418b-6a3e-44f6-b49a-4408dc108661/copy-manifests/0.log" Aug 04 01:01:27.067389 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:27.067364 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8594b76987-wqh9c_0708418b-6a3e-44f6-b49a-4408dc108661/manager/0.log" Aug 04 01:01:27.400405 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:27.400375 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5dc96dd749-lwtd6_8f795a71-27c6-40de-b980-e6cbfb421008/maas-api/0.log" Aug 04 01:01:28.689995 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:28.689950 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-l5jrb_93920fe4-ea33-49e7-965b-1dcfbebe35e8/manager/0.log" Aug 04 01:01:28.802279 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:28.802238 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-x9qlv_c6331656-1633-4dec-971b-aac2fec661f0/manager/0.log" Aug 04 01:01:28.912383 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:28.912352 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-jc6zh_c0d969fb-3209-4e74-87c9-713ec1c35cc1/kuadrant-console-plugin/0.log" Aug 04 01:01:29.024008 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:29.023935 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-8gqtg_9b2ce2d1-fe88-473c-a620-b5cd493fc65a/manager/0.log" Aug 04 01:01:29.127818 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:29.127768 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-qm2pq_440cc743-e375-4c81-835e-b954959c91a6/limitador/0.log" Aug 04 01:01:29.246408 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:29.246376 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-tvzb2_7c8667d8-9b61-43a5-8e6a-2eedd5372281/manager/0.log" Aug 04 01:01:29.911416 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:29.911387 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f7cf7b5f-xspfp_e129f437-cc33-4360-b132-927ae8b12083/kube-auth-proxy/0.log" Aug 04 01:01:30.021054 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:30.021022 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-g68xs_cb205534-bc74-4a36-b5cd-5ece6e5264ed/istio-proxy/1.log" Aug 04 01:01:30.674389 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:30.674362 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-kxxp2_7da78db6-0f82-4b0e-a846-025a6fb10f8f/main/0.log" Aug 04 01:01:30.886398 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:30.886367 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pn25d_86db44ce-bfd9-4848-8c56-0412cbff28d9/main/0.log" Aug 04 01:01:31.099830 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:31.099738 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-mb4j7_00355fc0-7beb-4357-8aec-62cec0b609ad/main/0.log" Aug 04 01:01:31.214863 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:31.214832 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-xhnbf_48efe6d7-3773-4953-8c70-493efb11aa65/main/0.log" Aug 04 01:01:36.094323 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:36.094288 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:36.193585 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:36.193551 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:38.566585 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:38.566550 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:38.668118 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:38.668088 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:01:38.909342 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:38.909311 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6k6zs_6bc834e8-f024-4f19-8b8f-b84b2f9d1f39/global-pull-secret-syncer/0.log" Aug 04 01:01:39.052181 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:39.052153 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" Aug 04 01:01:39.154740 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:39.154706 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-239.ec2.internal_3831fba3d380769b026bc2429ce14da9/haproxy/0.log" Aug 04 01:01:44.310434 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:44.310405 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-l5jrb_93920fe4-ea33-49e7-965b-1dcfbebe35e8/manager/0.log" Aug 04 01:01:44.340090 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:44.340058 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-x9qlv_c6331656-1633-4dec-971b-aac2fec661f0/manager/0.log" Aug 04 01:01:44.365364 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:44.365339 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-jc6zh_c0d969fb-3209-4e74-87c9-713ec1c35cc1/kuadrant-console-plugin/0.log" Aug 04 01:01:44.446699 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:44.446671 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-8gqtg_9b2ce2d1-fe88-473c-a620-b5cd493fc65a/manager/0.log" Aug 04 01:01:44.468885 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:44.468857 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-qm2pq_440cc743-e375-4c81-835e-b954959c91a6/limitador/0.log" Aug 04 01:01:44.547439 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:44.547408 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-tvzb2_7c8667d8-9b61-43a5-8e6a-2eedd5372281/manager/0.log" Aug 04 01:01:46.567485 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:46.567456 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-db9668795-zn7bm_6dc018cb-9822-44db-95af-a010165f4432/metrics-server/0.log" Aug 04 01:01:46.596479 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:46.596448 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-qqqfh_a8f5c806-cd10-4754-b0ee-626b79a38ff1/monitoring-plugin/0.log" Aug 04 01:01:46.794054 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:46.794030 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vzq8t_14616f46-049b-47c2-b8f3-8c65ae39308d/node-exporter/0.log" Aug 04 01:01:46.816523 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:46.816501 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vzq8t_14616f46-049b-47c2-b8f3-8c65ae39308d/kube-rbac-proxy/0.log" Aug 04 01:01:46.839087 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:46.839013 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vzq8t_14616f46-049b-47c2-b8f3-8c65ae39308d/init-textfile/0.log" Aug 04 01:01:47.246113 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:47.246085 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-tz6c9_32c81e49-2f55-4fc3-8cfe-1e41abd29d18/prometheus-operator-admission-webhook/0.log" Aug 04 01:01:48.671934 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:48.671909 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-cw62v_397af7b7-a308-43bc-b985-6b61dbc61347/networking-console-plugin/0.log" Aug 04 01:01:49.515508 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.515472 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4"] Aug 04 01:01:49.518125 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518099 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" Aug 04 01:01:49.518125 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518121 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518140 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518146 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518157 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f589d2cd-e4ae-493c-b671-0ec8e67ac817" containerName="maas-api" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518163 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f589d2cd-e4ae-493c-b671-0ec8e67ac817" containerName="maas-api" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518174 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518179 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518239 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f589d2cd-e4ae-493c-b671-0ec8e67ac817" containerName="maas-api" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518246 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518251 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="62deb26d-9ae2-4f5b-be38-3a6d6066ae10" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518258 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" Aug 04 01:01:49.518316 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.518263 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e83481ca-d176-4315-86e9-8f49027ed7fd" containerName="istio-proxy" Aug 04 01:01:49.521048 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.521030 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.523859 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.523837 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6jcsw\"/\"openshift-service-ca.crt\"" Aug 04 01:01:49.525026 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.525002 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6jcsw\"/\"kube-root-ca.crt\"" Aug 04 01:01:49.525157 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.525132 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6jcsw\"/\"default-dockercfg-jksch\"" Aug 04 01:01:49.525539 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.525518 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4"] Aug 04 01:01:49.609785 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.609754 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntsks\" (UniqueName: \"kubernetes.io/projected/c4162c94-fb5e-4590-b505-c5661ddf4463-kube-api-access-ntsks\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.609962 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.609823 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-podres\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.609962 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.609846 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-proc\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.609962 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.609863 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-lib-modules\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.610077 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.609966 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-sys\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711198 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711154 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-proc\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711198 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711201 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-lib-modules\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711675 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711285 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-proc\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711675 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711292 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-sys\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711675 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711337 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-sys\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711675 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711368 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntsks\" (UniqueName: \"kubernetes.io/projected/c4162c94-fb5e-4590-b505-c5661ddf4463-kube-api-access-ntsks\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711675 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711414 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-podres\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711675 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711439 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-lib-modules\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.711675 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.711519 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c4162c94-fb5e-4590-b505-c5661ddf4463-podres\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.723268 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.723244 2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntsks\" (UniqueName: \"kubernetes.io/projected/c4162c94-fb5e-4590-b505-c5661ddf4463-kube-api-access-ntsks\") pod \"perf-node-gather-daemonset-4xgq4\" (UID: \"c4162c94-fb5e-4590-b505-c5661ddf4463\") " pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.833003 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.832909 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:49.890725 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.890684 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-r577z_88a5d2f3-3361-4999-8823-d802b9f67fad/download-server/0.log" Aug 04 01:01:49.969998 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:49.969971 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4"] Aug 04 01:01:49.971968 ip-10-0-138-239 kubenswrapper[2551]: W0804 01:01:49.971927 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc4162c94_fb5e_4590_b505_c5661ddf4463.slice/crio-85d8038a57af6ced2a876d9ae756af2d646fb64d6e483a2a23703b03ecf8609d WatchSource:0}: Error finding container 85d8038a57af6ced2a876d9ae756af2d646fb64d6e483a2a23703b03ecf8609d: Status 404 returned error can't find the container with id 85d8038a57af6ced2a876d9ae756af2d646fb64d6e483a2a23703b03ecf8609d Aug 04 01:01:50.953635 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:50.953597 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" event={"ID":"c4162c94-fb5e-4590-b505-c5661ddf4463","Type":"ContainerStarted","Data":"3286132c510789f6e14a33e3b5ceab6a2a6dd06d3be0b8cfc70c500fc4db6f0c"} Aug 04 01:01:50.954104 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:50.953660 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" event={"ID":"c4162c94-fb5e-4590-b505-c5661ddf4463","Type":"ContainerStarted","Data":"85d8038a57af6ced2a876d9ae756af2d646fb64d6e483a2a23703b03ecf8609d"} Aug 04 01:01:50.954104 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:50.953903 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:50.969774 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:50.969722 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" podStartSLOduration=1.969706912 podStartE2EDuration="1.969706912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 01:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 01:01:50.968345793 +0000 UTC m=+3576.848793335" watchObservedRunningTime="2026-08-04 01:01:50.969706912 +0000 UTC m=+3576.850154453" Aug 04 01:01:51.746657 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:51.746629 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5nk69_2d3e5909-e3db-42e4-a285-a991937e28f5/dns/0.log" Aug 04 01:01:51.769902 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:51.769881 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5nk69_2d3e5909-e3db-42e4-a285-a991937e28f5/kube-rbac-proxy/0.log" Aug 04 01:01:51.894683 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:51.894656 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b5tk9_315bfe51-8619-4ecf-9e28-a1cdb0b5eebf/dns-node-resolver/0.log" Aug 04 01:01:52.495864 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:52.495834 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-85ffd74c56-qkg67_d5125ae2-f4a8-4d00-b77d-90af11295854/registry/0.log" Aug 04 01:01:52.525941 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:52.525915 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pgjn4_f3d39dda-92c3-44bf-b400-ab782afb57c3/node-ca/0.log" Aug 04 01:01:53.743684 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:53.743657 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f7cf7b5f-xspfp_e129f437-cc33-4360-b132-927ae8b12083/kube-auth-proxy/0.log" Aug 04 01:01:53.771784 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:53.771752 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-g68xs_cb205534-bc74-4a36-b5cd-5ece6e5264ed/istio-proxy/0.log" Aug 04 01:01:53.778413 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:53.778394 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-g68xs_cb205534-bc74-4a36-b5cd-5ece6e5264ed/istio-proxy/1.log" Aug 04 01:01:54.757335 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:54.757309 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sx2pp_dba2ddf7-7e20-424b-b5a3-06389a500ab7/serve-healthcheck-canary/0.log" Aug 04 01:01:55.389514 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:55.389484 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zpdc6_63919cf6-3390-422c-b39e-c7803a1654d6/kube-rbac-proxy/0.log" Aug 04 01:01:55.411279 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:55.411253 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zpdc6_63919cf6-3390-422c-b39e-c7803a1654d6/exporter/0.log" Aug 04 01:01:55.433078 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:55.433054 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zpdc6_63919cf6-3390-422c-b39e-c7803a1654d6/extractor/0.log" Aug 04 01:01:56.968355 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:56.968327 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6jcsw/perf-node-gather-daemonset-4xgq4" Aug 04 01:01:58.076286 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:58.076255 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-npdcp_2326363c-2143-4dec-8592-e5fdd7ab6dac/server/0.log" Aug 04 01:01:58.159493 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:58.159466 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2fvbv_bf451cea-1e4b-4100-9ebd-8dc323e00e92/manager/0.log" Aug 04 01:01:58.346445 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:58.346376 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8594b76987-wqh9c_0708418b-6a3e-44f6-b49a-4408dc108661/manager/0.log" Aug 04 01:01:58.368497 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:58.368475 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8594b76987-wqh9c_0708418b-6a3e-44f6-b49a-4408dc108661/copy-manifests/0.log" Aug 04 01:01:59.797879 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:59.797850 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f7cf7b5f-xspfp_e129f437-cc33-4360-b132-927ae8b12083/kube-auth-proxy/0.log" Aug 04 01:01:59.816337 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:59.816269 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-g68xs_cb205534-bc74-4a36-b5cd-5ece6e5264ed/istio-proxy/0.log" Aug 04 01:01:59.822714 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:59.822690 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-g68xs_cb205534-bc74-4a36-b5cd-5ece6e5264ed/istio-proxy/1.log" Aug 04 01:01:59.859751 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:01:59.859715 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-bcv99_d5399b99-3c84-457f-8018-837450a2920c/openshift-lws-operator/0.log" Aug 04 01:02:01.497466 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:01.497433 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57f7cf7b5f-xspfp_e129f437-cc33-4360-b132-927ae8b12083/kube-auth-proxy/0.log" Aug 04 01:02:01.519532 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:01.519500 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-g68xs_cb205534-bc74-4a36-b5cd-5ece6e5264ed/istio-proxy/0.log" Aug 04 01:02:01.525601 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:01.525580 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-g68xs_cb205534-bc74-4a36-b5cd-5ece6e5264ed/istio-proxy/1.log" Aug 04 01:02:03.733441 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:03.733403 2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4fr52_4f7ebb7f-20a8-43dd-b7f7-dd352ed466ba/konnectivity-agent/0.log" limit=1024 Aug 04 01:02:04.954978 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:04.954946 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-mndq6_016682c0-bacf-4a7b-a6f8-76cc9e7eff4b/migrator/0.log" Aug 04 01:02:04.975807 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:04.975769 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-mndq6_016682c0-bacf-4a7b-a6f8-76cc9e7eff4b/graceful-termination/0.log" Aug 04 01:02:06.657720 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:06.657693 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k5rzl_365c333a-8898-472b-8c70-21345d778a50/kube-multus-additional-cni-plugins/0.log" Aug 04 01:02:06.682948 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:06.682925 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k5rzl_365c333a-8898-472b-8c70-21345d778a50/egress-router-binary-copy/0.log" Aug 04 01:02:06.704276 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:06.704254 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k5rzl_365c333a-8898-472b-8c70-21345d778a50/cni-plugins/0.log" Aug 04 01:02:06.727677 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:06.727657 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k5rzl_365c333a-8898-472b-8c70-21345d778a50/bond-cni-plugin/0.log" Aug 04 01:02:06.748085 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:06.748064 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k5rzl_365c333a-8898-472b-8c70-21345d778a50/routeoverride-cni/0.log" Aug 04 01:02:06.768851 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:06.768820 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k5rzl_365c333a-8898-472b-8c70-21345d778a50/whereabouts-cni-bincopy/0.log" Aug 04 01:02:06.789932 ip-10-0-138-239 kubenswrapper[2551]: I0804 01:02:06.789912 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k5rzl_365c333a-8898-472b-8c70-21345d778a50/whereabouts-cni/0.log"