Jul 24 15:36:16.463610 ip-10-0-132-149 systemd[1]: Starting Kubernetes Kubelet... Jul 24 15:36:16.850383 ip-10-0-132-149 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 15:36:16.850383 ip-10-0-132-149 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 15:36:16.850383 ip-10-0-132-149 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 15:36:16.850383 ip-10-0-132-149 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 15:36:16.850383 ip-10-0-132-149 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 15:36:16.851218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.851124 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 15:36:16.858086 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858058 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:36:16.858086 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858080 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:36:16.858086 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858085 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:36:16.858086 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858089 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:36:16.858086 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858092 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:36:16.858086 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858096 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858100 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858106 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858111 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858117 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858122 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858126 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858130 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858134 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858137 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858140 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858145 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858148 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858152 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858155 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858160 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858163 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858167 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858170 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858175 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:36:16.858463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858178 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858182 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858186 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858190 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858193 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858197 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858210 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858214 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858220 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858226 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858231 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858236 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858239 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858244 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858249 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858254 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858259 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858264 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858268 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:36:16.859277 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858272 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858277 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858282 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858286 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858290 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858294 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858317 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858322 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858327 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858330 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858335 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858338 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858342 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858346 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858350 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858355 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858359 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858365 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858369 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:36:16.860114 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858373 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858377 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858381 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858385 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858389 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858393 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858398 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858406 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858413 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858418 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858423 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858427 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858431 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858436 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858440 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858444 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858448 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858452 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858456 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858460 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:36:16.860918 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858465 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858470 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858473 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858477 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858481 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858486 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858490 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858494 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858498 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858503 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858507 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858511 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858515 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858520 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858524 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.858528 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859713 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859723 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859728 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859732 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:36:16.861803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859738 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859742 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859747 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859751 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859755 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859759 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859763 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859767 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859772 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859775 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859780 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859786 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859792 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859798 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859802 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859807 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859811 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859815 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859819 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:36:16.862685 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859823 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859827 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859832 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859836 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859840 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859845 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859849 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859853 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859857 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859862 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859866 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859870 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859874 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859879 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859883 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859887 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859892 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859896 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859901 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859904 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:36:16.863371 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859909 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859913 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859918 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859922 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859926 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859930 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859934 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859939 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859943 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859947 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859951 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859955 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859959 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859963 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859968 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859973 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859977 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859981 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859985 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859989 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:36:16.863944 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859993 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.859997 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860001 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860006 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860010 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860014 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860018 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860027 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860033 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860037 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860042 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860046 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860050 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860055 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860059 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860063 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860067 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860072 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860078 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:36:16.864498 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860083 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860090 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860098 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860102 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860106 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860110 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860114 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860118 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860122 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860126 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860130 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860134 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860138 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860142 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860147 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860151 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860155 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860798 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860806 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:36:16.865063 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860811 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860815 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860819 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860824 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860828 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860833 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860837 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860841 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860845 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860850 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860854 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860858 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860862 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860866 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860871 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860878 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860884 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860889 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860898 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:36:16.865721 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860903 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860908 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860912 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860917 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860921 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860925 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860929 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860933 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860937 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860941 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860945 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860949 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860958 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860962 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860966 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860970 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860975 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860978 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860982 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860987 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:36:16.866187 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860991 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860994 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.860999 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861003 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861007 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861011 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861015 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861019 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861031 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861036 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861040 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861044 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861048 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861053 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861057 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861061 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861065 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861069 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861073 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861077 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:36:16.866774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861082 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861086 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861090 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861094 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861100 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861104 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861107 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861111 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861115 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861121 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861127 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861131 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861134 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861141 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861145 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861149 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861156 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861161 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861166 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:36:16.867327 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861171 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861175 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861180 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861184 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861188 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861192 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861197 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861201 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861205 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861209 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861213 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861217 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861221 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861225 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861229 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861233 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861237 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861244 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.861248 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861369 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 15:36:16.867788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861381 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861391 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861399 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861406 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861411 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861418 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861424 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861430 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861436 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861441 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861447 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861451 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861456 2560 flags.go:64] FLAG: --cgroup-root="" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861460 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861465 2560 flags.go:64] FLAG: --client-ca-file="" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861471 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861476 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861482 2560 flags.go:64] FLAG: --cluster-domain="" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861486 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861491 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861496 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861501 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861507 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 15:36:16.868273 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861512 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861517 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861522 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861527 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861531 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861536 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861543 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861547 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861554 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861559 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861563 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861568 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861573 2560 flags.go:64] FLAG: --enable-server="true" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861577 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861585 2560 flags.go:64] FLAG: --event-burst="100" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861590 2560 flags.go:64] FLAG: --event-qps="50" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861595 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861600 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861605 2560 flags.go:64] FLAG: --eviction-hard="" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861611 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861616 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861620 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861625 2560 flags.go:64] FLAG: --eviction-soft="" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861629 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861634 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 15:36:16.868878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861647 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861660 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861664 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861669 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861673 2560 flags.go:64] FLAG: --feature-gates="" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861679 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861684 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861689 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861694 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861699 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861704 2560 flags.go:64] FLAG: --help="false" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861708 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-132-149.ec2.internal" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861713 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861719 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861724 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861729 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861735 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861739 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861743 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861748 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861752 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861756 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861762 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861770 2560 flags.go:64] FLAG: --kube-reserved="" Jul 24 15:36:16.869577 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861775 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861779 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861784 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861789 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861794 2560 flags.go:64] FLAG: --lock-file="" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861798 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861802 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861806 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861814 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861823 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861828 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861833 2560 flags.go:64] FLAG: --logging-format="text" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861837 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861843 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861847 2560 flags.go:64] FLAG: --manifest-url="" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861852 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861859 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861864 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861870 2560 flags.go:64] FLAG: --max-pods="110" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861875 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861880 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861885 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861889 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861894 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861899 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 15:36:16.870198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861903 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861914 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861919 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861924 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861929 2560 flags.go:64] FLAG: --pod-cidr="" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861933 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861941 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861947 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861952 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861956 2560 flags.go:64] FLAG: --port="10250" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861961 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861966 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-010e478cf05cc8487" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861971 2560 flags.go:64] FLAG: --qos-reserved="" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861975 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861980 2560 flags.go:64] FLAG: --register-node="true" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861985 2560 flags.go:64] FLAG: --register-with-taints="" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.861990 2560 flags.go:64] FLAG: --registry-burst="10" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862003 2560 flags.go:64] FLAG: --registry-qps="5" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862008 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862013 2560 flags.go:64] FLAG: --reserved-memory="" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862022 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862027 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862032 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862036 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862041 2560 flags.go:64] FLAG: --runonce="false" Jul 24 15:36:16.870845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862046 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862051 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862055 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862060 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862065 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862070 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862074 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862079 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862084 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862088 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862093 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862098 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862102 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862107 2560 flags.go:64] FLAG: --system-cgroups="" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862114 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862122 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862126 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862131 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862141 2560 flags.go:64] FLAG: --tls-min-version="" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862145 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862150 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862154 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862159 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862163 2560 flags.go:64] FLAG: --v="2" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862169 2560 flags.go:64] FLAG: --version="false" Jul 24 15:36:16.871487 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862183 2560 flags.go:64] FLAG: --vmodule="" Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862190 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.862195 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862404 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862412 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862418 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862422 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862427 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862432 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862436 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862442 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862449 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862454 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862458 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862462 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862466 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862470 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862475 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862478 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862483 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:36:16.872099 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862488 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862494 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862498 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862502 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862506 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862510 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862514 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862518 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862522 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862526 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862530 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862534 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862538 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862543 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862547 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862551 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862555 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862559 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862563 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:36:16.872650 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862567 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862571 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862576 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862580 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862584 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862588 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862592 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862597 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862601 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862604 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862611 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862617 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862622 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862626 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862632 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862636 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862640 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862644 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862648 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:36:16.873103 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862652 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862656 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862660 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862665 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862669 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862673 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862677 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862681 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862686 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862689 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862693 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862697 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862701 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862704 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862708 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862712 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862718 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862723 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862727 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862731 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:36:16.873655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862735 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862740 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862744 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862748 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862751 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862755 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862760 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862767 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862771 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862775 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862779 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862783 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862787 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862791 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862795 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862801 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862806 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862810 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862814 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862818 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:36:16.874141 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862822 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862827 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862833 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.862838 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.863367 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.871196 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.871216 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.871230 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.871236 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871296 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871315 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871319 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871322 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871325 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:36:16.874655 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871328 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871331 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871333 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871336 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871339 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871343 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871346 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871349 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871352 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871354 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871357 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871360 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871362 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871365 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871367 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871370 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871372 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871375 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871377 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:36:16.875004 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871380 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871382 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871385 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871387 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871390 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871399 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871402 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871405 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871407 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871411 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871415 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871418 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871420 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871423 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871425 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871428 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871431 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871433 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871436 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871438 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:36:16.875492 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871441 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871443 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871446 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871448 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871450 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871453 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871455 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871457 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871460 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871462 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871464 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871467 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871469 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871471 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871475 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871477 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871481 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871484 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871493 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:36:16.875974 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871497 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871500 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871503 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871505 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871509 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871511 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871514 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871517 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871520 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871522 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871525 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871528 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871530 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871533 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871535 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871538 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871540 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871543 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871545 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:36:16.876463 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871548 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871550 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871552 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871555 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871557 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871560 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871562 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871565 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871568 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871570 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871573 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871575 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871578 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871586 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871589 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871591 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:36:16.876919 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871593 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.871599 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871747 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871753 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871756 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871759 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871762 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871764 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871766 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871769 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871771 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871774 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871776 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871778 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871781 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 15:36:16.877332 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871784 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871786 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871788 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871791 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871794 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871796 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871799 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871801 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871804 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871806 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871809 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871811 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871814 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871816 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871819 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871827 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871830 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871834 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871837 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 15:36:16.877702 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871840 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871843 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871845 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871847 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871850 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871852 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871854 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871858 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871862 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871865 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871868 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871871 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871873 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871876 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871878 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871880 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871883 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871885 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871888 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 15:36:16.878163 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871890 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871892 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871894 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871897 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871900 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871902 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871905 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871907 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871910 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871912 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871920 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871923 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871925 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871927 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871930 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871932 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871934 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871937 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871939 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871941 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 15:36:16.878647 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871943 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871946 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871948 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871950 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871953 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871955 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871957 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871960 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871962 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871965 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871967 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871969 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871972 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871974 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871977 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871980 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871982 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871985 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871987 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871990 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 15:36:16.879159 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871993 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871995 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.871997 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.872005 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.872008 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.872010 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.872012 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:16.872015 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.872019 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.872149 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.874245 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.875514 2560 server.go:1019] "Starting client certificate rotation" Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.875643 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 15:36:16.879921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.875681 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 15:36:16.880353 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.878120 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 15:36:16.882964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.882938 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 15:36:16.894683 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.894664 2560 log.go:25] "Validated CRI v1 runtime API" Jul 24 15:36:16.896621 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.896605 2560 log.go:25] "Validated CRI v1 image API" Jul 24 15:36:16.898122 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.898109 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 15:36:16.900364 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.900336 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 80f10463-1b2e-4cb9-9bed-a324a6e2efc9:/dev/nvme0n1p3 81dba2f3-4694-400c-9376-6dda51489cfb:/dev/nvme0n1p4] Jul 24 15:36:16.900444 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.900362 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 15:36:16.904533 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.904515 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 15:36:16.905774 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.905647 2560 manager.go:216] Machine: {Timestamp:2026-07-24 15:36:16.904293532 +0000 UTC m=+0.326894563 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099787 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b002f0d1b089e7a16aa4522fa15ed SystemUUID:ec2b002f-0d1b-089e-7a16-aa4522fa15ed BootID:cc41fe94-bfe0-46c9-9fd1-d8ffa980d155 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e2:eb:93:dd:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e2:eb:93:dd:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:0e:96:6a:b7:5a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 15:36:16.905774 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.905769 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 15:36:16.905886 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.905852 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 15:36:16.908978 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.908952 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 15:36:16.909117 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.908982 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-149.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 15:36:16.910229 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.910218 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 15:36:16.910261 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.910233 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 15:36:16.910261 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.910241 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 15:36:16.911416 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.911405 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 15:36:16.912564 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.912554 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 15:36:16.915954 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.915942 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 24 15:36:16.916092 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916082 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 15:36:16.916139 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916130 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 24 15:36:16.916168 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916144 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 15:36:16.916168 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916159 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 15:36:16.916243 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916168 2560 kubelet.go:398] "Adding apiserver pod source" Jul 24 15:36:16.916243 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916180 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 15:36:16.916767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916738 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 15:36:16.916767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.916766 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 15:36:16.918928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.918907 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 15:36:16.920221 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920207 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 15:36:16.920270 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920238 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 15:36:16.920799 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920788 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 15:36:16.920836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920805 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 15:36:16.920836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920812 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 15:36:16.920836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920820 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 15:36:16.920836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920830 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 15:36:16.920836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920836 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 15:36:16.920964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920844 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 15:36:16.920964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920850 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 15:36:16.920964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920856 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 15:36:16.920964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920861 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 15:36:16.920964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920869 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 15:36:16.920964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.920877 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 15:36:16.921561 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.921551 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 15:36:16.921619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.921562 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 15:36:16.925151 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.925135 2560 server.go:1268] "Started kubelet" Jul 24 15:36:16.925250 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.925212 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 15:36:16.925404 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.925362 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 15:36:16.925510 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.925435 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 15:36:16.926229 ip-10-0-132-149 systemd[1]: Started Kubernetes Kubelet. Jul 24 15:36:16.929142 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.929116 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 15:36:16.929244 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.929136 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xmvph" Jul 24 15:36:16.929538 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.929496 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-149.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 15:36:16.929940 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.929891 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 15:36:16.931593 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.931569 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 15:36:16.932333 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.932292 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-149.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 15:36:16.932576 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.932564 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 24 15:36:16.935046 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.935030 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xmvph" Jul 24 15:36:16.937040 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.937020 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 15:36:16.937161 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.937080 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 15:36:16.937161 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.937134 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:16.937592 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.937573 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 15:36:16.937592 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.937577 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 15:36:16.937733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.937602 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 15:36:16.937733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.937696 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 24 15:36:16.937733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.937707 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 24 15:36:16.938216 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938199 2560 factory.go:153] Registering CRI-O factory Jul 24 15:36:16.938287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938220 2560 factory.go:223] Registration of the crio container factory successfully Jul 24 15:36:16.938287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938275 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 15:36:16.938287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938284 2560 factory.go:55] Registering systemd factory Jul 24 15:36:16.938454 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938292 2560 factory.go:223] Registration of the systemd container factory successfully Jul 24 15:36:16.938454 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938328 2560 factory.go:103] Registering Raw factory Jul 24 15:36:16.938454 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938343 2560 manager.go:1196] Started watching for new ooms in manager Jul 24 15:36:16.938780 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.938765 2560 manager.go:349] Starting recovery of all containers Jul 24 15:36:16.941010 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.939999 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-149.ec2.internal.18c543cd6e985547 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-149.ec2.internal,UID:ip-10-0-132-149.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-149.ec2.internal,},FirstTimestamp:2026-07-24 15:36:16.925103431 +0000 UTC m=+0.347704451,LastTimestamp:2026-07-24 15:36:16.925103431 +0000 UTC m=+0.347704451,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-149.ec2.internal,}" Jul 24 15:36:16.943378 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.943339 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:36:16.946667 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.946486 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-149.ec2.internal\" not found" node="ip-10-0-132-149.ec2.internal" Jul 24 15:36:16.950526 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.950510 2560 manager.go:354] Recovery completed Jul 24 15:36:16.955029 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.955014 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:36:16.957033 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.957016 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:36:16.957113 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.957054 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:36:16.957113 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.957070 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:36:16.957595 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.957578 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 15:36:16.957681 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.957596 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 15:36:16.957681 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.957613 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 24 15:36:16.957681 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.957641 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:36:16.958426 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.958412 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:36:16.958484 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.958439 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:36:16.958484 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.958448 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:36:16.958646 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.958634 2560 policy_none.go:49] "None policy: Start" Jul 24 15:36:16.958679 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.958651 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 15:36:16.958679 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.958662 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 15:36:16.959692 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.959681 2560 policy_none.go:47] "Start" Jul 24 15:36:16.997961 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.997940 2560 manager.go:337] "Starting Device Plugin manager" Jul 24 15:36:16.998089 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.997981 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 15:36:16.998089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.997995 2560 server.go:87] "Starting device plugin registration server" Jul 24 15:36:16.998318 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.998286 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 15:36:16.998357 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.998325 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 15:36:16.998425 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.998411 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 15:36:16.998507 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.998496 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 15:36:16.998553 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:16.998508 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 15:36:16.999107 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.999086 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 15:36:16.999195 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:16.999137 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.098494 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.098460 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:36:17.099750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.099732 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:36:17.099813 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.099763 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:36:17.099813 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.099773 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:36:17.099813 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.099801 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.107841 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.107791 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.107841 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.107815 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-149.ec2.internal\": node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.122541 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.122516 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.133056 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.133031 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 15:36:17.134368 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.134353 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 15:36:17.134438 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.134387 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 15:36:17.134438 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.134410 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 15:36:17.134520 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.134441 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 15:36:17.139356 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.139333 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:36:17.222832 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.222801 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.234952 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.234926 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal"] Jul 24 15:36:17.235038 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.235028 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:36:17.237870 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.237852 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:36:17.237928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.237890 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:36:17.237928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.237907 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:36:17.239120 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.239098 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57bb7b3abac857e2c7619904cfa2d101-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal\" (UID: \"57bb7b3abac857e2c7619904cfa2d101\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.239195 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.239133 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57bb7b3abac857e2c7619904cfa2d101-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal\" (UID: \"57bb7b3abac857e2c7619904cfa2d101\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.239251 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.239238 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:36:17.239412 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.239389 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.239450 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.239428 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:36:17.240099 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.240081 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:36:17.240188 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.240107 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:36:17.240188 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.240125 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:36:17.240188 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.240167 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:36:17.240327 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.240197 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:36:17.240327 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.240215 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:36:17.241391 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.241373 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.241470 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.241406 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 15:36:17.242171 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.242153 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientMemory" Jul 24 15:36:17.242243 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.242186 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 15:36:17.242243 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.242198 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeHasSufficientPID" Jul 24 15:36:17.271156 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.271128 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-149.ec2.internal\" not found" node="ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.275765 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.275747 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-149.ec2.internal\" not found" node="ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.323532 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.323492 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.339369 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.339341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57bb7b3abac857e2c7619904cfa2d101-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal\" (UID: \"57bb7b3abac857e2c7619904cfa2d101\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.339449 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.339374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57bb7b3abac857e2c7619904cfa2d101-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal\" (UID: \"57bb7b3abac857e2c7619904cfa2d101\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.339449 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.339416 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fec91574dc3c4e18175b238dc0aeffc5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-149.ec2.internal\" (UID: \"fec91574dc3c4e18175b238dc0aeffc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.339525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.339447 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57bb7b3abac857e2c7619904cfa2d101-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal\" (UID: \"57bb7b3abac857e2c7619904cfa2d101\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.339525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.339461 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57bb7b3abac857e2c7619904cfa2d101-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal\" (UID: \"57bb7b3abac857e2c7619904cfa2d101\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.423833 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.423751 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.440096 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.440061 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fec91574dc3c4e18175b238dc0aeffc5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-149.ec2.internal\" (UID: \"fec91574dc3c4e18175b238dc0aeffc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.440178 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.440121 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fec91574dc3c4e18175b238dc0aeffc5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-149.ec2.internal\" (UID: \"fec91574dc3c4e18175b238dc0aeffc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.524277 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.524238 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.573754 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.573724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.579007 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.578983 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" Jul 24 15:36:17.625149 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.625119 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.725409 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.725367 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.762372 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.762314 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:36:17.826852 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.826806 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.875934 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.875889 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 15:36:17.876660 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.876081 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 15:36:17.876660 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.876096 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 15:36:17.876660 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.876096 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 15:36:17.927666 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:17.927620 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-149.ec2.internal\" not found" Jul 24 15:36:17.940129 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.940071 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 15:31:17 +0000 UTC" deadline="2028-04-17 03:44:28.069220528 +0000 UTC" Jul 24 15:36:17.940129 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.940130 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15180h8m10.129098862s" Jul 24 15:36:17.963654 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:17.963581 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:36:18.038108 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.038081 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" Jul 24 15:36:18.051185 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.051161 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 15:36:18.052318 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.052301 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" Jul 24 15:36:18.061835 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.061815 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 15:36:18.158208 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:18.158159 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57bb7b3abac857e2c7619904cfa2d101.slice/crio-af17681324843612b07f16476337636809019c722e5b9253adf78dbabc9c26af WatchSource:0}: Error finding container af17681324843612b07f16476337636809019c722e5b9253adf78dbabc9c26af: Status 404 returned error can't find the container with id af17681324843612b07f16476337636809019c722e5b9253adf78dbabc9c26af Jul 24 15:36:18.163359 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.163343 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:36:18.708027 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.707996 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:36:18.919665 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.919616 2560 apiserver.go:52] "Watching apiserver" Jul 24 15:36:18.928452 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.928421 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 15:36:18.928884 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.928858 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-xwm8x","kube-system/global-pull-secret-syncer-hk86b","kube-system/konnectivity-agent-ldrdb","openshift-cluster-node-tuning-operator/tuned-b5fgd","openshift-dns/node-resolver-gbwxh","openshift-image-registry/node-ca-76tzw","openshift-multus/multus-additional-cni-plugins-7jjqs","openshift-multus/multus-t8wsn","openshift-network-diagnostics/network-check-target-xcghk","kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal","openshift-multus/network-metrics-daemon-7ffzk","openshift-network-operator/iptables-alerter-z5p9j"] Jul 24 15:36:18.931337 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.931295 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.932488 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.932465 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:18.932606 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:18.932550 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:18.933666 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.933648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:18.934193 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.934145 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n6wnm\"" Jul 24 15:36:18.934193 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.934151 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 15:36:18.934193 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.934172 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 15:36:18.934372 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.934162 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 15:36:18.934557 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.934541 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 15:36:18.934618 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.934589 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 15:36:18.934821 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.934776 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.936019 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.936002 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:18.936288 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.936228 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 15:36:18.936288 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.936256 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 15:36:18.936492 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.936481 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wsn8d\"" Jul 24 15:36:18.937067 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.937048 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:18.937169 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.937108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-km2d8\"" Jul 24 15:36:18.937169 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.937161 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 15:36:18.937672 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.937652 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 15:36:18.938274 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.938255 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 15:36:18.938274 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.938269 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l7vfx\"" Jul 24 15:36:18.938500 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.938481 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 15:36:18.938591 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.938555 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 15:36:18.938647 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.938599 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:18.938834 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:18.938667 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:18.939618 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.939599 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b9n24\"" Jul 24 15:36:18.939742 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.939702 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 15:36:18.940413 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.939950 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 15:36:18.940413 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.940012 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:18.940413 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.940084 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 15:36:18.941635 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.941616 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:18.941723 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:18.941692 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:18.942138 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.942106 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 15:36:18.943085 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.943068 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:18.943186 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.943106 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 15:36:18.943186 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.943123 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 15:36:18.943555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.943534 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 15:36:18.943555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.943551 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qj62g\"" Jul 24 15:36:18.943714 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.943569 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 15:36:18.943714 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.943568 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 15:36:18.944742 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.944726 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:18.946223 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.946195 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 15:36:18.946483 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.946437 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 15:36:18.946557 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.946484 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8q6l9\"" Jul 24 15:36:18.946631 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.946611 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 15:36:18.947642 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.947622 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 15:36:18.947746 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.947688 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vglbh\"" Jul 24 15:36:18.947912 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.947890 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948226 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t8wsn" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948288 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b556c24d-9aee-4fd8-ab2c-9605f1300488-hosts-file\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b556c24d-9aee-4fd8-ab2c-9605f1300488-tmp-dir\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948400 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vql6w\" (UniqueName: \"kubernetes.io/projected/105c143a-fbc6-4945-8311-5e9be7b5ce68-kube-api-access-vql6w\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948418 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff3a54e8-eddc-46d6-bec0-994078145a1b-agent-certs\") pod \"konnectivity-agent-ldrdb\" (UID: \"ff3a54e8-eddc-46d6-bec0-994078145a1b\") " pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysctl-conf\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948503 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-run\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-tuned\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-modprobe-d\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948629 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-lib-modules\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:18.948767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948774 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2dvk\" (UniqueName: \"kubernetes.io/projected/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-kube-api-access-g2dvk\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff3a54e8-eddc-46d6-bec0-994078145a1b-konnectivity-ca\") pod \"konnectivity-agent-ldrdb\" (UID: \"ff3a54e8-eddc-46d6-bec0-994078145a1b\") " pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-var-lib-kubelet\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.948945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/105c143a-fbc6-4945-8311-5e9be7b5ce68-host\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949002 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-system-cni-dir\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-cnibin\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949101 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-os-release\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-systemd\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-sys\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949255 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-host\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949296 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949378 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w49q7\" (UniqueName: \"kubernetes.io/projected/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-kube-api-access-w49q7\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-tmp\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.949599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949520 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-cni-binary-copy\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.950015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949609 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.950015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949657 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvjv\" (UniqueName: \"kubernetes.io/projected/b1451fa7-d70f-468e-9aea-828d9ea60632-kube-api-access-kzvjv\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:18.950015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysconfig\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.950015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949770 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-kubernetes\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.950015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqk8t\" (UniqueName: \"kubernetes.io/projected/b556c24d-9aee-4fd8-ab2c-9605f1300488-kube-api-access-qqk8t\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:18.950015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949873 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysctl-d\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:18.950015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.949900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/105c143a-fbc6-4945-8311-5e9be7b5ce68-serviceca\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:18.950945 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.950923 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 15:36:18.950945 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.950936 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fdrpd\"" Jul 24 15:36:18.951094 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.950931 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 15:36:18.977752 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.977715 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-td88j" Jul 24 15:36:18.991756 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:18.991728 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-td88j" Jul 24 15:36:19.038269 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.038232 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 15:36:19.050198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-tuned\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.050347 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-modprobe-d\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.050347 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050242 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-daemon-config\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.050347 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050267 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-etc-kubernetes\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.050347 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-systemd\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.050347 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-ovnkube-config\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjrcl\" (UniqueName: \"kubernetes.io/projected/6ec7e935-2018-4645-9147-4cbea13123dd-kube-api-access-xjrcl\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050397 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-modprobe-d\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050393 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-cni-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-os-release\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa213600-00b6-44d8-9f86-cbad3d6be7ae-cni-binary-copy\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050541 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff3a54e8-eddc-46d6-bec0-994078145a1b-konnectivity-ca\") pod \"konnectivity-agent-ldrdb\" (UID: \"ff3a54e8-eddc-46d6-bec0-994078145a1b\") " pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050569 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-var-lib-kubelet\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-socket-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.050639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050600 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.050658 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050725 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-var-lib-kubelet\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.050742 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs podName:a8239cd3-c74a-4738-9eaf-4b55ce5dc184 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:19.550717983 +0000 UTC m=+2.973318996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs") pod "network-metrics-daemon-7ffzk" (UID: "a8239cd3-c74a-4738-9eaf-4b55ce5dc184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-device-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050794 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-k8s-cni-cncf-io\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050819 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-conf-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050863 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-cnibin\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-os-release\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-system-cni-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050919 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81dbe12d-777c-4e87-b2e8-e002792f7435-ovn-node-metrics-cert\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-cnibin\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.050990 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-os-release\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.051036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w49q7\" (UniqueName: \"kubernetes.io/projected/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-kube-api-access-w49q7\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051050 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-netns\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-multus-certs\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-var-lib-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051162 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-ovn\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051169 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff3a54e8-eddc-46d6-bec0-994078145a1b-konnectivity-ca\") pod \"konnectivity-agent-ldrdb\" (UID: \"ff3a54e8-eddc-46d6-bec0-994078145a1b\") " pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051192 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-run-ovn-kubernetes\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/96502b52-d0fc-49f9-b839-8008f7b76266-kubelet-config\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051291 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ec7e935-2018-4645-9147-4cbea13123dd-iptables-alerter-script\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-tmp\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-cni-binary-copy\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysconfig\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051482 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqk8t\" (UniqueName: \"kubernetes.io/projected/b556c24d-9aee-4fd8-ab2c-9605f1300488-kube-api-access-qqk8t\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051526 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-log-socket\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051555 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/105c143a-fbc6-4945-8311-5e9be7b5ce68-serviceca\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051570 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysconfig\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.051830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051582 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-etc-selinux\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051618 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-kubelet\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-run-netns\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051672 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-etc-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051699 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-slash\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051756 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-ovnkube-script-lib\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff3a54e8-eddc-46d6-bec0-994078145a1b-agent-certs\") pod \"konnectivity-agent-ldrdb\" (UID: \"ff3a54e8-eddc-46d6-bec0-994078145a1b\") " pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051814 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysctl-conf\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-socket-dir-parent\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051877 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-lib-modules\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-sys-fs\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.051956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-cni-bin\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052013 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2dvk\" (UniqueName: \"kubernetes.io/projected/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-kube-api-access-g2dvk\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/105c143a-fbc6-4945-8311-5e9be7b5ce68-host\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.052909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052082 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/105c143a-fbc6-4945-8311-5e9be7b5ce68-serviceca\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052099 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-kubelet\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052115 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-cni-binary-copy\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/96502b52-d0fc-49f9-b839-8008f7b76266-dbus\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052138 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-cni-multus\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052209 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/105c143a-fbc6-4945-8311-5e9be7b5ce68-host\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052240 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-system-cni-dir\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysctl-conf\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-systemd\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052290 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-sys\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052337 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-systemd\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052334 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-lib-modules\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1451fa7-d70f-468e-9aea-828d9ea60632-system-cni-dir\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052451 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-sys\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-host\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.053885 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.052969 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053008 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq6gx\" (UniqueName: \"kubernetes.io/projected/fa213600-00b6-44d8-9f86-cbad3d6be7ae-kube-api-access-tq6gx\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053035 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-cni-netd\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053061 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-node-log\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053063 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-host\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-env-overrides\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ec7e935-2018-4645-9147-4cbea13123dd-host-slash\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053150 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvjv\" (UniqueName: \"kubernetes.io/projected/b1451fa7-d70f-468e-9aea-828d9ea60632-kube-api-access-kzvjv\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-kubernetes\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-registration-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-hostroot\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysctl-d\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053243 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-kubernetes\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b556c24d-9aee-4fd8-ab2c-9605f1300488-hosts-file\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b556c24d-9aee-4fd8-ab2c-9605f1300488-tmp-dir\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vql6w\" (UniqueName: \"kubernetes.io/projected/105c143a-fbc6-4945-8311-5e9be7b5ce68-kube-api-access-vql6w\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:19.054689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-sysctl-d\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053362 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b62b\" (UniqueName: \"kubernetes.io/projected/3b628822-394d-4bb6-ad20-b429eb4f8e98-kube-api-access-9b62b\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053406 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-cni-bin\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053417 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b556c24d-9aee-4fd8-ab2c-9605f1300488-hosts-file\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2wv2\" (UniqueName: \"kubernetes.io/projected/81dbe12d-777c-4e87-b2e8-e002792f7435-kube-api-access-v2wv2\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-run\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-cnibin\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053504 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-systemd-units\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053643 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-run\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.053789 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b556c24d-9aee-4fd8-ab2c-9605f1300488-tmp-dir\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.054545 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1451fa7-d70f-468e-9aea-828d9ea60632-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.054741 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-etc-tuned\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.054788 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-tmp\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.055457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.055030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff3a54e8-eddc-46d6-bec0-994078145a1b-agent-certs\") pod \"konnectivity-agent-ldrdb\" (UID: \"ff3a54e8-eddc-46d6-bec0-994078145a1b\") " pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:19.061802 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.061775 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:36:19.061937 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.061810 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:36:19.061937 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.061828 2560 projected.go:196] Error preparing data for projected volume kube-api-access-x7gnk for pod openshift-network-diagnostics/network-check-target-xcghk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:19.061937 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.061898 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk podName:549ce8b0-a24a-421b-b074-8391efed6999 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:19.561877162 +0000 UTC m=+2.984478238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x7gnk" (UniqueName: "kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk") pod "network-check-target-xcghk" (UID: "549ce8b0-a24a-421b-b074-8391efed6999") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:19.063479 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.063451 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w49q7\" (UniqueName: \"kubernetes.io/projected/22b789cc-80c5-40a4-b02e-d5f8c3a0595c-kube-api-access-w49q7\") pod \"tuned-b5fgd\" (UID: \"22b789cc-80c5-40a4-b02e-d5f8c3a0595c\") " pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.063590 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.063461 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2dvk\" (UniqueName: \"kubernetes.io/projected/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-kube-api-access-g2dvk\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:19.063590 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.063574 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vql6w\" (UniqueName: \"kubernetes.io/projected/105c143a-fbc6-4945-8311-5e9be7b5ce68-kube-api-access-vql6w\") pod \"node-ca-76tzw\" (UID: \"105c143a-fbc6-4945-8311-5e9be7b5ce68\") " pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:19.063958 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.063938 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvjv\" (UniqueName: \"kubernetes.io/projected/b1451fa7-d70f-468e-9aea-828d9ea60632-kube-api-access-kzvjv\") pod \"multus-additional-cni-plugins-7jjqs\" (UID: \"b1451fa7-d70f-468e-9aea-828d9ea60632\") " pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.064185 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.064166 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqk8t\" (UniqueName: \"kubernetes.io/projected/b556c24d-9aee-4fd8-ab2c-9605f1300488-kube-api-access-qqk8t\") pod \"node-resolver-gbwxh\" (UID: \"b556c24d-9aee-4fd8-ab2c-9605f1300488\") " pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:19.087153 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.087125 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec91574dc3c4e18175b238dc0aeffc5.slice/crio-a1be5df59998c3bd4fa7232d32ffa23603ec287648ab756edeae41d1e927177b WatchSource:0}: Error finding container a1be5df59998c3bd4fa7232d32ffa23603ec287648ab756edeae41d1e927177b: Status 404 returned error can't find the container with id a1be5df59998c3bd4fa7232d32ffa23603ec287648ab756edeae41d1e927177b Jul 24 15:36:19.138839 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.138783 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" event={"ID":"fec91574dc3c4e18175b238dc0aeffc5","Type":"ContainerStarted","Data":"a1be5df59998c3bd4fa7232d32ffa23603ec287648ab756edeae41d1e927177b"} Jul 24 15:36:19.140000 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.139973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" event={"ID":"57bb7b3abac857e2c7619904cfa2d101","Type":"ContainerStarted","Data":"af17681324843612b07f16476337636809019c722e5b9253adf78dbabc9c26af"} Jul 24 15:36:19.154757 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-env-overrides\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.154818 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ec7e935-2018-4645-9147-4cbea13123dd-host-slash\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.154818 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-registration-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.154818 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-hostroot\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.154960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154821 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b62b\" (UniqueName: \"kubernetes.io/projected/3b628822-394d-4bb6-ad20-b429eb4f8e98-kube-api-access-9b62b\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.154960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-cni-bin\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.154960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154860 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ec7e935-2018-4645-9147-4cbea13123dd-host-slash\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.154960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2wv2\" (UniqueName: \"kubernetes.io/projected/81dbe12d-777c-4e87-b2e8-e002792f7435-kube-api-access-v2wv2\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.154960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154899 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-cnibin\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.154960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154919 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-hostroot\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.154960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154948 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-cnibin\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154949 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-systemd-units\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154991 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-daemon-config\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-etc-kubernetes\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-systemd\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.154897 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-registration-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155060 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-systemd-units\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155060 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-ovnkube-config\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-cni-bin\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155101 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjrcl\" (UniqueName: \"kubernetes.io/projected/6ec7e935-2018-4645-9147-4cbea13123dd-kube-api-access-xjrcl\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155113 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-etc-kubernetes\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-cni-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155188 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-systemd\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155258 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-os-release\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155276 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-cni-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.155308 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155190 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-os-release\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa213600-00b6-44d8-9f86-cbad3d6be7ae-cni-binary-copy\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-socket-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155433 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-env-overrides\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-device-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-k8s-cni-cncf-io\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155509 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-device-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-conf-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155580 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-socket-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155586 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-system-cni-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155624 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-k8s-cni-cncf-io\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155627 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-ovnkube-config\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-system-cni-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155676 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-conf-dir\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155681 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81dbe12d-777c-4e87-b2e8-e002792f7435-ovn-node-metrics-cert\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.156201 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155769 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-netns\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155787 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-daemon-config\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.155829 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155832 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-netns\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-multus-certs\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.155893 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret podName:96502b52-d0fc-49f9-b839-8008f7b76266 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:19.655873061 +0000 UTC m=+3.078474092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret") pod "global-pull-secret-syncer-hk86b" (UID: "96502b52-d0fc-49f9-b839-8008f7b76266") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155908 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-run-multus-certs\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-var-lib-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-ovn\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155960 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa213600-00b6-44d8-9f86-cbad3d6be7ae-cni-binary-copy\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-run-ovn-kubernetes\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.155992 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-var-lib-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/96502b52-d0fc-49f9-b839-8008f7b76266-kubelet-config\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ec7e935-2018-4645-9147-4cbea13123dd-iptables-alerter-script\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156048 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-run-ovn\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156053 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-run-ovn-kubernetes\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156075 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-log-socket\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.156932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156106 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/96502b52-d0fc-49f9-b839-8008f7b76266-kubelet-config\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156108 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-etc-selinux\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-kubelet\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156174 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-kubelet\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-log-socket\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156236 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-run-netns\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-etc-selinux\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-etc-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-slash\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156298 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-etc-openvswitch\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-run-netns\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-ovnkube-script-lib\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156384 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-slash\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156398 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-socket-dir-parent\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156425 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-sys-fs\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.157480 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-cni-bin\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-kubelet\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156516 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-multus-socket-dir-parent\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/96502b52-d0fc-49f9-b839-8008f7b76266-dbus\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156600 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ec7e935-2018-4645-9147-4cbea13123dd-iptables-alerter-script\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156618 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-kubelet\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-sys-fs\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3b628822-394d-4bb6-ad20-b429eb4f8e98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156718 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-cni-bin\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-cni-multus\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156775 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fa213600-00b6-44d8-9f86-cbad3d6be7ae-host-var-lib-cni-multus\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156779 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/96502b52-d0fc-49f9-b839-8008f7b76266-dbus\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156802 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq6gx\" (UniqueName: \"kubernetes.io/projected/fa213600-00b6-44d8-9f86-cbad3d6be7ae-kube-api-access-tq6gx\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-cni-netd\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-node-log\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156907 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-node-log\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.156915 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81dbe12d-777c-4e87-b2e8-e002792f7435-host-cni-netd\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158016 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.157088 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81dbe12d-777c-4e87-b2e8-e002792f7435-ovnkube-script-lib\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.158561 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.158466 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81dbe12d-777c-4e87-b2e8-e002792f7435-ovn-node-metrics-cert\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.164868 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.164827 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2wv2\" (UniqueName: \"kubernetes.io/projected/81dbe12d-777c-4e87-b2e8-e002792f7435-kube-api-access-v2wv2\") pod \"ovnkube-node-xwm8x\" (UID: \"81dbe12d-777c-4e87-b2e8-e002792f7435\") " pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.165021 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.165000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq6gx\" (UniqueName: \"kubernetes.io/projected/fa213600-00b6-44d8-9f86-cbad3d6be7ae-kube-api-access-tq6gx\") pod \"multus-t8wsn\" (UID: \"fa213600-00b6-44d8-9f86-cbad3d6be7ae\") " pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.165499 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.165481 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b62b\" (UniqueName: \"kubernetes.io/projected/3b628822-394d-4bb6-ad20-b429eb4f8e98-kube-api-access-9b62b\") pod \"aws-ebs-csi-driver-node-z95p4\" (UID: \"3b628822-394d-4bb6-ad20-b429eb4f8e98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.165760 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.165742 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjrcl\" (UniqueName: \"kubernetes.io/projected/6ec7e935-2018-4645-9147-4cbea13123dd-kube-api-access-xjrcl\") pod \"iptables-alerter-z5p9j\" (UID: \"6ec7e935-2018-4645-9147-4cbea13123dd\") " pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.243041 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.242958 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" Jul 24 15:36:19.251998 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.251978 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:19.252774 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.252749 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1451fa7_d70f_468e_9aea_828d9ea60632.slice/crio-ec16d4117b3e6b5e508e6f31af56336b6380b06e2493abfc2ea2c6f5923e3dce WatchSource:0}: Error finding container ec16d4117b3e6b5e508e6f31af56336b6380b06e2493abfc2ea2c6f5923e3dce: Status 404 returned error can't find the container with id ec16d4117b3e6b5e508e6f31af56336b6380b06e2493abfc2ea2c6f5923e3dce Jul 24 15:36:19.259219 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.259194 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" Jul 24 15:36:19.260749 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.260724 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3a54e8_eddc_46d6_bec0_994078145a1b.slice/crio-4ce70052203aa534fb648c234e2c0fd6504c56dbb380d9c1525df45440c86e79 WatchSource:0}: Error finding container 4ce70052203aa534fb648c234e2c0fd6504c56dbb380d9c1525df45440c86e79: Status 404 returned error can't find the container with id 4ce70052203aa534fb648c234e2c0fd6504c56dbb380d9c1525df45440c86e79 Jul 24 15:36:19.262857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.262839 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbwxh" Jul 24 15:36:19.268842 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.268816 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-76tzw" Jul 24 15:36:19.269975 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.269945 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b789cc_80c5_40a4_b02e_d5f8c3a0595c.slice/crio-eaca8a81267cf5c5d3712502f3b709e83f9b766ce7fc703f904680ced6d38b49 WatchSource:0}: Error finding container eaca8a81267cf5c5d3712502f3b709e83f9b766ce7fc703f904680ced6d38b49: Status 404 returned error can't find the container with id eaca8a81267cf5c5d3712502f3b709e83f9b766ce7fc703f904680ced6d38b49 Jul 24 15:36:19.275105 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.274552 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb556c24d_9aee_4fd8_ab2c_9605f1300488.slice/crio-f31b1a5d4e2cef9c129ed737c20d198dad5836914dd8e98a31d8302d8cf331e6 WatchSource:0}: Error finding container f31b1a5d4e2cef9c129ed737c20d198dad5836914dd8e98a31d8302d8cf331e6: Status 404 returned error can't find the container with id f31b1a5d4e2cef9c129ed737c20d198dad5836914dd8e98a31d8302d8cf331e6 Jul 24 15:36:19.275105 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.275078 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:19.281287 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.281261 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105c143a_fbc6_4945_8311_5e9be7b5ce68.slice/crio-3de98e5889aa600d2f3da389836eb455066f84e0893d8ecd78d0d4c7828c25d5 WatchSource:0}: Error finding container 3de98e5889aa600d2f3da389836eb455066f84e0893d8ecd78d0d4c7828c25d5: Status 404 returned error can't find the container with id 3de98e5889aa600d2f3da389836eb455066f84e0893d8ecd78d0d4c7828c25d5 Jul 24 15:36:19.281394 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.281316 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z5p9j" Jul 24 15:36:19.284705 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.284677 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81dbe12d_777c_4e87_b2e8_e002792f7435.slice/crio-99b6b35fb55a487ce6b591fa329ed6e9a04d526eada6c9a0543160b87c35de39 WatchSource:0}: Error finding container 99b6b35fb55a487ce6b591fa329ed6e9a04d526eada6c9a0543160b87c35de39: Status 404 returned error can't find the container with id 99b6b35fb55a487ce6b591fa329ed6e9a04d526eada6c9a0543160b87c35de39 Jul 24 15:36:19.288029 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.288007 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" Jul 24 15:36:19.290061 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.290041 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ec7e935_2018_4645_9147_4cbea13123dd.slice/crio-70e3f48a7c582f6bfa8eff529c2da1a7101740901ef2d43149e5e0679a361411 WatchSource:0}: Error finding container 70e3f48a7c582f6bfa8eff529c2da1a7101740901ef2d43149e5e0679a361411: Status 404 returned error can't find the container with id 70e3f48a7c582f6bfa8eff529c2da1a7101740901ef2d43149e5e0679a361411 Jul 24 15:36:19.292570 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.292547 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t8wsn" Jul 24 15:36:19.296421 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.296358 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b628822_394d_4bb6_ad20_b429eb4f8e98.slice/crio-7ccc1f8a38fdf9ad7fa7b1b3884229025aa9bd70af1ab0b77c1b5b6c1119dfd0 WatchSource:0}: Error finding container 7ccc1f8a38fdf9ad7fa7b1b3884229025aa9bd70af1ab0b77c1b5b6c1119dfd0: Status 404 returned error can't find the container with id 7ccc1f8a38fdf9ad7fa7b1b3884229025aa9bd70af1ab0b77c1b5b6c1119dfd0 Jul 24 15:36:19.302771 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:36:19.302749 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa213600_00b6_44d8_9f86_cbad3d6be7ae.slice/crio-67ae554eb1bfd39abc731f5091eb5efcee8bbe93e1c63018bd09ba69c5f2bd7f WatchSource:0}: Error finding container 67ae554eb1bfd39abc731f5091eb5efcee8bbe93e1c63018bd09ba69c5f2bd7f: Status 404 returned error can't find the container with id 67ae554eb1bfd39abc731f5091eb5efcee8bbe93e1c63018bd09ba69c5f2bd7f Jul 24 15:36:19.468589 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.468562 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:36:19.559580 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.559485 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:19.559747 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.559693 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:19.559825 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.559762 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs podName:a8239cd3-c74a-4738-9eaf-4b55ce5dc184 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:20.559745378 +0000 UTC m=+3.982346404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs") pod "network-metrics-daemon-7ffzk" (UID: "a8239cd3-c74a-4738-9eaf-4b55ce5dc184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:19.660729 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.660690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:19.660883 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.660752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:19.660936 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.660891 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:19.660978 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.660968 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret podName:96502b52-d0fc-49f9-b839-8008f7b76266 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:20.660948293 +0000 UTC m=+4.083549306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret") pod "global-pull-secret-syncer-hk86b" (UID: "96502b52-d0fc-49f9-b839-8008f7b76266") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:19.661085 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.661032 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:36:19.661085 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.661091 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:36:19.661331 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.661122 2560 projected.go:196] Error preparing data for projected volume kube-api-access-x7gnk for pod openshift-network-diagnostics/network-check-target-xcghk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:19.661331 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:19.661210 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk podName:549ce8b0-a24a-421b-b074-8391efed6999 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:20.661191262 +0000 UTC m=+4.083792293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x7gnk" (UniqueName: "kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk") pod "network-check-target-xcghk" (UID: "549ce8b0-a24a-421b-b074-8391efed6999") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:19.994751 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.994701 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:31:19 +0000 UTC" deadline="2028-02-25 10:53:15.129637251 +0000 UTC" Jul 24 15:36:19.994751 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:19.994743 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13939h16m55.13489914s" Jul 24 15:36:20.135225 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.135131 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:20.135442 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.135278 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:20.135769 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.135746 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:20.135886 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.135863 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:20.148692 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.148652 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ldrdb" event={"ID":"ff3a54e8-eddc-46d6-bec0-994078145a1b","Type":"ContainerStarted","Data":"4ce70052203aa534fb648c234e2c0fd6504c56dbb380d9c1525df45440c86e79"} Jul 24 15:36:20.156404 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.156364 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerStarted","Data":"ec16d4117b3e6b5e508e6f31af56336b6380b06e2493abfc2ea2c6f5923e3dce"} Jul 24 15:36:20.162002 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.161959 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z5p9j" event={"ID":"6ec7e935-2018-4645-9147-4cbea13123dd","Type":"ContainerStarted","Data":"70e3f48a7c582f6bfa8eff529c2da1a7101740901ef2d43149e5e0679a361411"} Jul 24 15:36:20.167612 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.167569 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"99b6b35fb55a487ce6b591fa329ed6e9a04d526eada6c9a0543160b87c35de39"} Jul 24 15:36:20.169959 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.169924 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbwxh" event={"ID":"b556c24d-9aee-4fd8-ab2c-9605f1300488","Type":"ContainerStarted","Data":"f31b1a5d4e2cef9c129ed737c20d198dad5836914dd8e98a31d8302d8cf331e6"} Jul 24 15:36:20.175280 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.175234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" event={"ID":"22b789cc-80c5-40a4-b02e-d5f8c3a0595c","Type":"ContainerStarted","Data":"eaca8a81267cf5c5d3712502f3b709e83f9b766ce7fc703f904680ced6d38b49"} Jul 24 15:36:20.186601 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.186556 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t8wsn" event={"ID":"fa213600-00b6-44d8-9f86-cbad3d6be7ae","Type":"ContainerStarted","Data":"67ae554eb1bfd39abc731f5091eb5efcee8bbe93e1c63018bd09ba69c5f2bd7f"} Jul 24 15:36:20.198470 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.198427 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" event={"ID":"3b628822-394d-4bb6-ad20-b429eb4f8e98","Type":"ContainerStarted","Data":"7ccc1f8a38fdf9ad7fa7b1b3884229025aa9bd70af1ab0b77c1b5b6c1119dfd0"} Jul 24 15:36:20.200924 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.200896 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-76tzw" event={"ID":"105c143a-fbc6-4945-8311-5e9be7b5ce68","Type":"ContainerStarted","Data":"3de98e5889aa600d2f3da389836eb455066f84e0893d8ecd78d0d4c7828c25d5"} Jul 24 15:36:20.369964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.366701 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 15:36:20.572354 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.572238 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:20.572543 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.572495 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:20.572610 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.572573 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs podName:a8239cd3-c74a-4738-9eaf-4b55ce5dc184 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:22.572550585 +0000 UTC m=+5.995151619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs") pod "network-metrics-daemon-7ffzk" (UID: "a8239cd3-c74a-4738-9eaf-4b55ce5dc184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:20.673485 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.673440 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:20.673689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.673527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:20.673770 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.673707 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:36:20.673770 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.673728 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:36:20.673770 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.673742 2560 projected.go:196] Error preparing data for projected volume kube-api-access-x7gnk for pod openshift-network-diagnostics/network-check-target-xcghk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:20.673930 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.673815 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk podName:549ce8b0-a24a-421b-b074-8391efed6999 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:22.673792732 +0000 UTC m=+6.096393766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x7gnk" (UniqueName: "kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk") pod "network-check-target-xcghk" (UID: "549ce8b0-a24a-421b-b074-8391efed6999") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:20.674348 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.674242 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:20.674348 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:20.674305 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret podName:96502b52-d0fc-49f9-b839-8008f7b76266 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:22.674288611 +0000 UTC m=+6.096889676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret") pod "global-pull-secret-syncer-hk86b" (UID: "96502b52-d0fc-49f9-b839-8008f7b76266") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:20.996099 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.995970 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:31:19 +0000 UTC" deadline="2027-12-31 07:21:08.928778445 +0000 UTC" Jul 24 15:36:20.996099 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:20.996038 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12591h44m47.932744276s" Jul 24 15:36:21.135372 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:21.135041 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:21.135597 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:21.135500 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:22.136441 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:22.136398 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:22.136946 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.136552 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:22.136946 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:22.136636 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:22.136946 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.136711 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:22.595482 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:22.595388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:22.595660 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.595581 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:22.595660 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.595654 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs podName:a8239cd3-c74a-4738-9eaf-4b55ce5dc184 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:26.595633138 +0000 UTC m=+10.018234160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs") pod "network-metrics-daemon-7ffzk" (UID: "a8239cd3-c74a-4738-9eaf-4b55ce5dc184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:22.696569 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:22.696526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:22.696779 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:22.696619 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:22.696779 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.696750 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:22.696908 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.696819 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret podName:96502b52-d0fc-49f9-b839-8008f7b76266 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:26.696799016 +0000 UTC m=+10.119400040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret") pod "global-pull-secret-syncer-hk86b" (UID: "96502b52-d0fc-49f9-b839-8008f7b76266") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:22.697289 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.697267 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:36:22.697411 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.697295 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:36:22.697411 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.697317 2560 projected.go:196] Error preparing data for projected volume kube-api-access-x7gnk for pod openshift-network-diagnostics/network-check-target-xcghk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:22.697411 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:22.697369 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk podName:549ce8b0-a24a-421b-b074-8391efed6999 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:26.697352772 +0000 UTC m=+10.119953784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x7gnk" (UniqueName: "kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk") pod "network-check-target-xcghk" (UID: "549ce8b0-a24a-421b-b074-8391efed6999") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:23.138014 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:23.137576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:23.138014 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:23.137725 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:24.135513 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:24.135480 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:24.135760 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:24.135618 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:24.135989 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:24.135956 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:24.136138 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:24.136081 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:25.135562 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:25.135523 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:25.136085 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:25.135695 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:26.135980 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:26.135265 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:26.135980 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.135425 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:26.135980 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:26.135513 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:26.135980 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.135591 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:26.634240 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:26.634195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:26.634476 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.634377 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:26.634476 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.634470 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs podName:a8239cd3-c74a-4738-9eaf-4b55ce5dc184 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:34.63444544 +0000 UTC m=+18.057046465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs") pod "network-metrics-daemon-7ffzk" (UID: "a8239cd3-c74a-4738-9eaf-4b55ce5dc184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:26.734890 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:26.734852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:26.735075 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:26.734916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:26.735075 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.735040 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:26.735075 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.735052 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:36:26.735075 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.735077 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:36:26.735315 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.735092 2560 projected.go:196] Error preparing data for projected volume kube-api-access-x7gnk for pod openshift-network-diagnostics/network-check-target-xcghk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:26.735315 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.735125 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret podName:96502b52-d0fc-49f9-b839-8008f7b76266 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:34.735101954 +0000 UTC m=+18.157702970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret") pod "global-pull-secret-syncer-hk86b" (UID: "96502b52-d0fc-49f9-b839-8008f7b76266") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:26.735315 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:26.735150 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk podName:549ce8b0-a24a-421b-b074-8391efed6999 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:34.735133907 +0000 UTC m=+18.157734918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x7gnk" (UniqueName: "kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk") pod "network-check-target-xcghk" (UID: "549ce8b0-a24a-421b-b074-8391efed6999") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:27.136472 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:27.136436 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:27.136912 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:27.136566 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:28.135457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:28.135425 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:28.135639 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:28.135563 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:28.135639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:28.135628 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:28.135734 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:28.135681 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:29.138023 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:29.137480 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:29.138023 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:29.137648 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:30.134893 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:30.134844 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:30.134893 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:30.134894 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:30.135145 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:30.134988 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:30.135200 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:30.135138 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:31.135181 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:31.135142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:31.135677 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:31.135316 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:32.135318 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:32.135266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:32.135752 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:32.135336 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:32.135752 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:32.135424 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:32.135752 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:32.135558 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:33.135272 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:33.135227 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:33.135488 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:33.135372 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:34.135053 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:34.135021 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:34.135251 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:34.135020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:34.135251 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.135135 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:34.135251 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.135210 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:34.698289 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:34.698246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:34.698830 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.698432 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:34.698830 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.698502 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs podName:a8239cd3-c74a-4738-9eaf-4b55ce5dc184 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:50.69848395 +0000 UTC m=+34.121084959 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs") pod "network-metrics-daemon-7ffzk" (UID: "a8239cd3-c74a-4738-9eaf-4b55ce5dc184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:34.799571 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:34.799504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:34.799571 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:34.799573 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:34.799797 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.799671 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:34.799797 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.799713 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:36:34.799797 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.799732 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:36:34.799797 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.799747 2560 projected.go:196] Error preparing data for projected volume kube-api-access-x7gnk for pod openshift-network-diagnostics/network-check-target-xcghk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:34.799797 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.799746 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret podName:96502b52-d0fc-49f9-b839-8008f7b76266 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:50.799726285 +0000 UTC m=+34.222327291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret") pod "global-pull-secret-syncer-hk86b" (UID: "96502b52-d0fc-49f9-b839-8008f7b76266") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:34.800026 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:34.799810 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk podName:549ce8b0-a24a-421b-b074-8391efed6999 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:50.799792194 +0000 UTC m=+34.222393202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x7gnk" (UniqueName: "kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk") pod "network-check-target-xcghk" (UID: "549ce8b0-a24a-421b-b074-8391efed6999") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:35.135059 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:35.135022 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:35.135242 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:35.135169 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:36.135248 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:36.135210 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:36.135691 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:36.135257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:36.135691 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:36.135343 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:36.135691 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:36.135494 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:37.138446 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:37.137082 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:37.138446 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:37.137256 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:38.135656 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:38.135619 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:38.135843 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:38.135747 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:38.135843 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:38.135811 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:38.135961 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:38.135928 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:39.135195 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:39.135158 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:39.135763 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:39.135325 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:40.135744 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.135560 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:40.136342 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.135601 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:40.136342 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:40.135839 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:40.136342 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:40.135879 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:40.256144 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.256111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t8wsn" event={"ID":"fa213600-00b6-44d8-9f86-cbad3d6be7ae","Type":"ContainerStarted","Data":"01bf40f26628a3764355efed6bfb0738852bc5de91ab56270cca99d3cb4579e2"} Jul 24 15:36:40.257360 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.257335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" event={"ID":"fec91574dc3c4e18175b238dc0aeffc5","Type":"ContainerStarted","Data":"8bae70d4d68be83ca70ecd3362b00c42da142e238643da13787fc677ae11f404"} Jul 24 15:36:40.259435 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.259411 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"1fa085f753bb5f2163e461171d18cb92d46cb93061130a0593917a2040cc2006"} Jul 24 15:36:40.259539 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.259488 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"ee4d33a8fa11c137e757df57c9b83c64aff43351842f86be20b0bf6f4cd9391e"} Jul 24 15:36:40.259603 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.259543 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"f351faff467ef492d4ab1dd5836d48eef1c1b29d31c05262dc9fcf6570854ef1"} Jul 24 15:36:40.260641 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.260612 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" event={"ID":"22b789cc-80c5-40a4-b02e-d5f8c3a0595c","Type":"ContainerStarted","Data":"60e897fc328fd38d71b026aad8ce1ba951dbd4e2e9762a9ead08502f704c7da8"} Jul 24 15:36:40.277954 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.277895 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b5fgd" podStartSLOduration=2.709278384 podStartE2EDuration="23.277880495s" totalImagesPullingTime="20.568602111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:40.277865848 +0000 UTC m=+23.700466895" watchObservedRunningTime="2026-07-24 15:36:40.277880495 +0000 UTC m=+23.700481536" Jul 24 15:36:40.295151 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.295110 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t8wsn" podStartSLOduration=2.724595882 podStartE2EDuration="23.295094301s" totalImagesPullingTime="20.570498419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:40.29489838 +0000 UTC m=+23.717499421" watchObservedRunningTime="2026-07-24 15:36:40.295094301 +0000 UTC m=+23.717695332" Jul 24 15:36:40.314083 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:40.314037 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-149.ec2.internal" podStartSLOduration=22.314021163 podStartE2EDuration="22.314021163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:40.313589514 +0000 UTC m=+23.736190562" watchObservedRunningTime="2026-07-24 15:36:40.314021163 +0000 UTC m=+23.736622212" Jul 24 15:36:41.135408 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.135191 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:41.135608 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:41.135520 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:41.265351 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.265297 2560 generic.go:356] "Generic (PLEG): container finished" podID="57bb7b3abac857e2c7619904cfa2d101" containerID="ae159eab213a8b3a54d4ec1f64b6cac5f50452cc30db700565fc3339c4911f9f" exitCode=0 Jul 24 15:36:41.265990 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.265402 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" event={"ID":"57bb7b3abac857e2c7619904cfa2d101","Type":"ContainerDied","Data":"ae159eab213a8b3a54d4ec1f64b6cac5f50452cc30db700565fc3339c4911f9f"} Jul 24 15:36:41.267232 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.267202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" event={"ID":"3b628822-394d-4bb6-ad20-b429eb4f8e98","Type":"ContainerStarted","Data":"87c7cfd21f8b544841ebeb191a5ea4dc5211b56b60d35d075f347fcd310c1b08"} Jul 24 15:36:41.269089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.269061 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-76tzw" event={"ID":"105c143a-fbc6-4945-8311-5e9be7b5ce68","Type":"ContainerStarted","Data":"969e09adcdf06dfca52e0f34e2352c748f53acf3169cf8724943d7b8def5c323"} Jul 24 15:36:41.270339 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.270294 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ldrdb" event={"ID":"ff3a54e8-eddc-46d6-bec0-994078145a1b","Type":"ContainerStarted","Data":"31281f5b3a0ed5e6569ab802d3d8b7a02757ffce84b106a3ab423f448ba924ea"} Jul 24 15:36:41.271769 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.271747 2560 generic.go:356] "Generic (PLEG): container finished" podID="b1451fa7-d70f-468e-9aea-828d9ea60632" containerID="234428db30d6c33d7181572e21201d67ff86e023fed64a8df2ad69b39c9d5b7f" exitCode=0 Jul 24 15:36:41.271875 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.271821 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerDied","Data":"234428db30d6c33d7181572e21201d67ff86e023fed64a8df2ad69b39c9d5b7f"} Jul 24 15:36:41.274986 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.274964 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"4ea7288cfd363ef3c9406269bdaa70c6c662399cc88ceffec652aa66afb22928"} Jul 24 15:36:41.275060 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.275024 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"e9888b1c92460dfe1de586d80da5c427dd1daa3314c4ced552899be72fc408d1"} Jul 24 15:36:41.275103 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.275071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"b25ce285cb9e208f5581397810bab31bd7a9d50b2c0b81f1e071df55b78f8a9e"} Jul 24 15:36:41.276282 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.276239 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbwxh" event={"ID":"b556c24d-9aee-4fd8-ab2c-9605f1300488","Type":"ContainerStarted","Data":"6a37b5e2b6a88743efd51ec2874918b275bd298bf456d5df0e6c1bf3f4821b42"} Jul 24 15:36:41.297293 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.297239 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ldrdb" podStartSLOduration=3.7483983260000002 podStartE2EDuration="24.297222089s" totalImagesPullingTime="20.548823763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:41.29689734 +0000 UTC m=+24.719498390" watchObservedRunningTime="2026-07-24 15:36:41.297222089 +0000 UTC m=+24.719823137" Jul 24 15:36:41.310470 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.310422 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-76tzw" podStartSLOduration=3.782246459 podStartE2EDuration="24.310405808s" totalImagesPullingTime="20.528159349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:41.309904011 +0000 UTC m=+24.732505041" watchObservedRunningTime="2026-07-24 15:36:41.310405808 +0000 UTC m=+24.733006847" Jul 24 15:36:41.346472 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.346418 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gbwxh" podStartSLOduration=3.808551127 podStartE2EDuration="24.346401513s" totalImagesPullingTime="20.537850386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:41.346142394 +0000 UTC m=+24.768743439" watchObservedRunningTime="2026-07-24 15:36:41.346401513 +0000 UTC m=+24.769002547" Jul 24 15:36:41.397342 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.397252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:41.476574 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.476538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:41.771764 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:41.771740 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 15:36:42.017745 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.017649 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T15:36:41.771759136Z","UUID":"d2fdf304-a6f4-4a0c-b139-da7999a400a4","Handler":null,"Name":"","Endpoint":""} Jul 24 15:36:42.019523 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.019501 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 15:36:42.019625 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.019540 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 15:36:42.135379 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.135346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:42.135608 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.135357 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:42.135608 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:42.135480 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:42.135608 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:42.135544 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:42.261221 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.261180 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:42.261837 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.261819 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:42.280940 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.280863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" event={"ID":"3b628822-394d-4bb6-ad20-b429eb4f8e98","Type":"ContainerStarted","Data":"c0061d3d94fe8a9442feb148a86a1367090af0fa811b750dfe029201c4757f0e"} Jul 24 15:36:42.282376 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.282344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z5p9j" event={"ID":"6ec7e935-2018-4645-9147-4cbea13123dd","Type":"ContainerStarted","Data":"cca4c14029171a4d7109b50424325bdd6475d0d272367c35c750e00296aa6608"} Jul 24 15:36:42.284097 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.284070 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" event={"ID":"57bb7b3abac857e2c7619904cfa2d101","Type":"ContainerStarted","Data":"beffaafa8d4c2b095ffedb893a262b208b854d21017f76214b6689de314740b3"} Jul 24 15:36:42.301558 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.301496 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z5p9j" podStartSLOduration=4.782381524 podStartE2EDuration="25.301474964s" totalImagesPullingTime="20.51909344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:42.301105408 +0000 UTC m=+25.723706459" watchObservedRunningTime="2026-07-24 15:36:42.301474964 +0000 UTC m=+25.724076008" Jul 24 15:36:42.315100 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.315045 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-149.ec2.internal" podStartSLOduration=24.315025836 podStartE2EDuration="24.315025836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:42.314586123 +0000 UTC m=+25.737187166" watchObservedRunningTime="2026-07-24 15:36:42.315025836 +0000 UTC m=+25.737626864" Jul 24 15:36:42.326744 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.326707 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:42.327411 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.327380 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ldrdb" Jul 24 15:36:42.419367 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.419325 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:42.520181 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:42.520146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:43.138878 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:43.138848 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:43.139116 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:43.138979 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:43.290045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:43.289996 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"2568eba259279cfce89448f975e394c28830a076ee82455e7ffd9a81ac7295ba"} Jul 24 15:36:43.518643 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:43.518612 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:43.573351 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:43.573294 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:44.135726 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:44.135685 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:44.135976 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:44.135742 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:44.135976 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:44.135835 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:44.135976 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:44.135955 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:44.297473 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:44.297427 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" event={"ID":"3b628822-394d-4bb6-ad20-b429eb4f8e98","Type":"ContainerStarted","Data":"05edbf34ebb49e3b401980acd0a3a38bcf9d1716b6a2c5815d3e3fba949f2913"} Jul 24 15:36:44.314482 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:44.314417 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z95p4" podStartSLOduration=4.148561048 podStartE2EDuration="27.314400714s" totalImagesPullingTime="23.165839666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:44.314180544 +0000 UTC m=+27.736781591" watchObservedRunningTime="2026-07-24 15:36:44.314400714 +0000 UTC m=+27.737001770" Jul 24 15:36:45.135091 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:45.134996 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:45.135256 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:45.135136 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:46.135149 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.134920 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:46.135801 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.134920 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:46.135801 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:46.135192 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:46.135801 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:46.135248 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:46.306554 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.306517 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" event={"ID":"81dbe12d-777c-4e87-b2e8-e002792f7435","Type":"ContainerStarted","Data":"1d0cd55d11ff500d7d697fe4bd4df1bd51b2c3bdce0081e3aa35eb864ba1f1c5"} Jul 24 15:36:46.306786 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.306764 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:46.306882 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.306861 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:46.308784 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.308756 2560 generic.go:356] "Generic (PLEG): container finished" podID="b1451fa7-d70f-468e-9aea-828d9ea60632" containerID="dd112a8de4b23357128c4c790eddcc8635ae6b9dcb1ca0431cfbad5ae4ddf732" exitCode=0 Jul 24 15:36:46.308893 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.308797 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerDied","Data":"dd112a8de4b23357128c4c790eddcc8635ae6b9dcb1ca0431cfbad5ae4ddf732"} Jul 24 15:36:46.324283 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.324254 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:46.324474 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.324459 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:46.332498 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.332462 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" podStartSLOduration=8.720731495999999 podStartE2EDuration="29.332448904s" totalImagesPullingTime="20.611717408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:46.330922353 +0000 UTC m=+29.753523426" watchObservedRunningTime="2026-07-24 15:36:46.332448904 +0000 UTC m=+29.755049930" Jul 24 15:36:46.609114 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.609077 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:46.708210 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.708178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:46.807396 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:46.807363 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:47.138042 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.138016 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:47.138734 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:47.138115 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:47.311785 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.311595 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:36:47.603419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.603382 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:47.682440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.682411 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:47.901750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.901719 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:47.910011 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.909981 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hk86b"] Jul 24 15:36:47.910259 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.910125 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:47.910259 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:47.910236 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:47.911140 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.911118 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xcghk"] Jul 24 15:36:47.911252 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.911219 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:47.911370 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:47.911351 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:47.911794 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.911777 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7ffzk"] Jul 24 15:36:47.911872 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.911853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:47.911956 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:47.911940 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:47.991585 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:47.991558 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:48.093339 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.093285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:48.316043 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.316004 2560 generic.go:356] "Generic (PLEG): container finished" podID="b1451fa7-d70f-468e-9aea-828d9ea60632" containerID="aa13b393f84974b5137ce9a20458405eeeb8529da44950d875e746a46b03019f" exitCode=0 Jul 24 15:36:48.316592 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.316094 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerDied","Data":"aa13b393f84974b5137ce9a20458405eeeb8529da44950d875e746a46b03019f"} Jul 24 15:36:48.368529 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.368445 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:48.431204 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.431169 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:48.543034 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.543000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:48.605601 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.605564 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:48.807694 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:48.807648 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:49.093482 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:49.093405 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:49.135212 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:49.135184 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:49.135403 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:49.135333 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:49.190235 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:49.190202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:50.135366 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.135330 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:50.135366 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.135372 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:50.135831 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.135458 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:50.135831 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.135586 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:50.323643 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.323606 2560 generic.go:356] "Generic (PLEG): container finished" podID="b1451fa7-d70f-468e-9aea-828d9ea60632" containerID="b0e0721eddf382e2d6a029972af7336ab52aa8948ef0884fdbe860bab67d5d09" exitCode=0 Jul 24 15:36:50.323794 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.323660 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerDied","Data":"b0e0721eddf382e2d6a029972af7336ab52aa8948ef0884fdbe860bab67d5d09"} Jul 24 15:36:50.692454 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.692431 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbwxh_b556c24d-9aee-4fd8-ab2c-9605f1300488/dns-node-resolver/0.log" Jul 24 15:36:50.717606 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.717565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:50.717774 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.717687 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:50.717774 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.717739 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs podName:a8239cd3-c74a-4738-9eaf-4b55ce5dc184 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:22.717726031 +0000 UTC m=+66.140327038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs") pod "network-metrics-daemon-7ffzk" (UID: "a8239cd3-c74a-4738-9eaf-4b55ce5dc184") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 15:36:50.733898 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.733865 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:50.818250 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.818222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:50.818454 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.818265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:50.818454 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.818405 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 15:36:50.818454 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.818405 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:50.818454 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.818419 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 15:36:50.818454 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.818431 2560 projected.go:196] Error preparing data for projected volume kube-api-access-x7gnk for pod openshift-network-diagnostics/network-check-target-xcghk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:50.818658 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.818474 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret podName:96502b52-d0fc-49f9-b839-8008f7b76266 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:22.818457134 +0000 UTC m=+66.241058142 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret") pod "global-pull-secret-syncer-hk86b" (UID: "96502b52-d0fc-49f9-b839-8008f7b76266") : object "kube-system"/"original-pull-secret" not registered Jul 24 15:36:50.818658 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:50.818489 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk podName:549ce8b0-a24a-421b-b074-8391efed6999 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:22.818483146 +0000 UTC m=+66.241084153 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x7gnk" (UniqueName: "kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk") pod "network-check-target-xcghk" (UID: "549ce8b0-a24a-421b-b074-8391efed6999") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 15:36:50.830159 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:50.830121 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:51.134865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:51.134836 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:51.135046 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:51.134981 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:51.656337 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:51.655541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:51.689665 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:51.689634 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-76tzw_105c143a-fbc6-4945-8311-5e9be7b5ce68/node-ca/0.log" Jul 24 15:36:51.861867 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:51.861825 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:51.919134 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:51.919099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:52.134901 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:52.134819 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:52.134901 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:52.134851 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:52.135117 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:52.134950 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:52.135117 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:52.135070 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:53.135558 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:53.135287 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:53.136027 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:53.135664 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:54.135021 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:54.134979 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:54.135021 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:54.135019 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:54.135255 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:54.135139 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:54.135341 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:54.135266 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:54.561454 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:54.561423 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:55.135357 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.135323 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:55.135536 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:55.135436 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:55.235994 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.235958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:55.338067 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.338025 2560 generic.go:356] "Generic (PLEG): container finished" podID="b1451fa7-d70f-468e-9aea-828d9ea60632" containerID="d51ce105589d30a69d475b3a68b42289b45e6ee236c639bfe45446d7f70bcfe0" exitCode=0 Jul 24 15:36:55.338258 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.338102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerDied","Data":"d51ce105589d30a69d475b3a68b42289b45e6ee236c639bfe45446d7f70bcfe0"} Jul 24 15:36:55.359904 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.359838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:55.483469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.483431 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:55.609096 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.609057 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:55.711321 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.711224 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:55.895415 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:55.895382 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:56.135341 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:56.135287 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:56.135341 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:56.135333 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:56.135561 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:56.135429 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:56.135561 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:56.135517 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:56.342940 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:56.342904 2560 generic.go:356] "Generic (PLEG): container finished" podID="b1451fa7-d70f-468e-9aea-828d9ea60632" containerID="6e39c52ede1902caa8b61d4eb1a61c029005e803bdeae9b4c7fc70ce31f4f5ac" exitCode=0 Jul 24 15:36:56.343103 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:56.342943 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerDied","Data":"6e39c52ede1902caa8b61d4eb1a61c029005e803bdeae9b4c7fc70ce31f4f5ac"} Jul 24 15:36:56.685814 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:56.685712 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:56.747225 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:56.747186 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:56.978162 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:56.978077 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:57.035879 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.035846 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:57.135948 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.135913 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:57.136093 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:57.136000 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:57.297227 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.297193 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:57.352550 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.352510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" event={"ID":"b1451fa7-d70f-468e-9aea-828d9ea60632","Type":"ContainerStarted","Data":"e3edfca936cb58f4fce7cf76150c4861c427135900e7b835485140a3707b69b8"} Jul 24 15:36:57.379051 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.379000 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7jjqs" podStartSLOduration=9.061770162 podStartE2EDuration="40.3789847s" totalImagesPullingTime="31.317214538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:57.378726109 +0000 UTC m=+40.801327153" watchObservedRunningTime="2026-07-24 15:36:57.3789847 +0000 UTC m=+40.801585729" Jul 24 15:36:57.498175 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.498142 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:57.695849 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.695766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:57.926898 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:57.926850 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:58.034717 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:58.034680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:58.135078 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:58.135043 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:36:58.135233 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:58.135043 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:36:58.135233 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:58.135171 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:36:58.135356 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:58.135229 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:36:58.813452 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:58.813408 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:58.914258 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:58.914224 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:59.093246 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:59.093166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:59.135249 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:59.135220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:36:59.135431 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:36:59.135373 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:36:59.297466 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:59.297431 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:59.492677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:59.492592 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:59.698126 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:59.698092 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:59.798564 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:59.798535 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:59.924511 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:36:59.924478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:00.135142 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:00.135049 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:00.135142 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:00.135087 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:00.135336 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:00.135177 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:00.135336 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:00.135262 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:01.041974 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.041941 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:01.134965 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.134927 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:01.135138 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:01.135041 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:37:01.143161 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.143130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:01.329943 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.329849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:01.504277 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.504245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:01.566479 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.566446 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:01.694860 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.694779 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:01.892667 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:01.892627 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:02.135064 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:02.135032 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:02.135507 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:02.135148 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:02.135507 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:02.135244 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:02.135507 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:02.135360 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:03.091601 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:03.091562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:03.134697 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:03.134667 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:03.134872 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:03.134770 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:37:03.301415 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:03.301379 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:03.501862 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:03.501777 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:03.693320 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:03.693269 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:03.906497 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:03.906461 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:04.135060 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:04.135030 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:04.135235 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:04.135166 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:04.135235 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:04.135222 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:04.135357 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:04.135338 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:05.099803 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:05.099769 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:05.135073 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:05.135032 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:05.135267 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:05.135140 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:37:05.310948 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:05.310913 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:05.501625 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:05.501546 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:05.694950 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:05.694916 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:05.903974 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:05.903945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:06.134834 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:06.134802 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:06.135338 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:06.134815 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:06.135338 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:06.134904 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:06.135338 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:06.135014 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:07.136020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:07.135982 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:07.136420 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:07.136075 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:37:08.135316 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:08.135262 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:08.135520 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:08.135263 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:08.135520 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:08.135391 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:08.135520 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:08.135476 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:09.135241 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:09.135208 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:09.135833 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:09.135332 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:37:10.134682 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:10.134634 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:10.134682 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:10.134682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:10.134896 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:10.134761 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:10.134896 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:10.134868 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:11.134710 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:11.134674 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:11.135088 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:11.134806 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:37:11.671838 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:11.671800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:11.773080 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:11.773043 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:12.135547 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:12.135508 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:12.135547 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:12.135546 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:12.136002 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:12.135646 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:12.136002 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:12.135804 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:13.134951 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:13.134910 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:13.135128 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:13.135020 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xcghk" podUID="549ce8b0-a24a-421b-b074-8391efed6999" Jul 24 15:37:14.135218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.135177 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:14.135218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.135208 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:14.135743 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:14.135294 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ffzk" podUID="a8239cd3-c74a-4738-9eaf-4b55ce5dc184" Jul 24 15:37:14.135743 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:14.135426 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hk86b" podUID="96502b52-d0fc-49f9-b839-8008f7b76266" Jul 24 15:37:14.164921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.164889 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:14.248718 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.248685 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:14.345809 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.345777 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:14.378679 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.378659 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-149.ec2.internal" event="NodeReady" Jul 24 15:37:14.378797 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.378783 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 15:37:14.446595 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.446516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:14.449271 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.449246 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-84w5l"] Jul 24 15:37:14.451186 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.451170 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.454828 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.454803 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 15:37:14.455194 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.455174 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 15:37:14.455282 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.455255 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 15:37:14.455631 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.455613 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 15:37:14.455825 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.455805 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9pmnt\"" Jul 24 15:37:14.456929 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.456911 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mqhw4"] Jul 24 15:37:14.458718 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.458698 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.461965 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.461945 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 15:37:14.462049 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.461980 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 15:37:14.462049 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.462009 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r6s28\"" Jul 24 15:37:14.475910 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.475886 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-84w5l"] Jul 24 15:37:14.478296 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.478270 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mqhw4"] Jul 24 15:37:14.485481 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n6q6\" (UniqueName: \"kubernetes.io/projected/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-kube-api-access-7n6q6\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.485604 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485500 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4eafb17-8bbc-42ec-8284-0330af851b55-data-volume\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.485604 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-tmp-dir\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.485604 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4eafb17-8bbc-42ec-8284-0330af851b55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.485722 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485610 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-config-volume\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.485722 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4eafb17-8bbc-42ec-8284-0330af851b55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.485722 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-metrics-tls\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.485722 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp72t\" (UniqueName: \"kubernetes.io/projected/c4eafb17-8bbc-42ec-8284-0330af851b55-kube-api-access-bp72t\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.485883 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.485769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4eafb17-8bbc-42ec-8284-0330af851b55-crio-socket\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.559748 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.559711 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8dj77"] Jul 24 15:37:14.561828 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.561808 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.562712 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.562688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:14.565521 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.565503 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 15:37:14.565592 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.565524 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lrd6q\"" Jul 24 15:37:14.565592 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.565538 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 15:37:14.565837 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.565823 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 15:37:14.574209 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.574179 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dj77"] Jul 24 15:37:14.586787 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.586756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n6q6\" (UniqueName: \"kubernetes.io/projected/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-kube-api-access-7n6q6\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.586938 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.586793 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e689b39-eaaf-4e95-86f9-9018172c2f7c-cert\") pod \"ingress-canary-8dj77\" (UID: \"1e689b39-eaaf-4e95-86f9-9018172c2f7c\") " pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.586938 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.586819 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4eafb17-8bbc-42ec-8284-0330af851b55-data-volume\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.586938 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.586837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-tmp-dir\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.586938 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.586899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hprz6\" (UniqueName: \"kubernetes.io/projected/1e689b39-eaaf-4e95-86f9-9018172c2f7c-kube-api-access-hprz6\") pod \"ingress-canary-8dj77\" (UID: \"1e689b39-eaaf-4e95-86f9-9018172c2f7c\") " pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.587150 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.586995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4eafb17-8bbc-42ec-8284-0330af851b55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.587150 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-config-volume\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.587150 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587057 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4eafb17-8bbc-42ec-8284-0330af851b55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.587150 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-metrics-tls\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.587345 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4eafb17-8bbc-42ec-8284-0330af851b55-data-volume\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.587345 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp72t\" (UniqueName: \"kubernetes.io/projected/c4eafb17-8bbc-42ec-8284-0330af851b55-kube-api-access-bp72t\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.587451 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4eafb17-8bbc-42ec-8284-0330af851b55-crio-socket\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.587499 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587457 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4eafb17-8bbc-42ec-8284-0330af851b55-crio-socket\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.587615 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587593 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4eafb17-8bbc-42ec-8284-0330af851b55-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.587672 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587636 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-tmp-dir\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.587741 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.587715 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-config-volume\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.591130 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.591107 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4eafb17-8bbc-42ec-8284-0330af851b55-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.591206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.591166 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-metrics-tls\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.600716 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.600679 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n6q6\" (UniqueName: \"kubernetes.io/projected/c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae-kube-api-access-7n6q6\") pod \"dns-default-mqhw4\" (UID: \"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae\") " pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.601025 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.601004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp72t\" (UniqueName: \"kubernetes.io/projected/c4eafb17-8bbc-42ec-8284-0330af851b55-kube-api-access-bp72t\") pod \"insights-runtime-extractor-84w5l\" (UID: \"c4eafb17-8bbc-42ec-8284-0330af851b55\") " pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.688245 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.688199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hprz6\" (UniqueName: \"kubernetes.io/projected/1e689b39-eaaf-4e95-86f9-9018172c2f7c-kube-api-access-hprz6\") pod \"ingress-canary-8dj77\" (UID: \"1e689b39-eaaf-4e95-86f9-9018172c2f7c\") " pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.688477 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.688342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e689b39-eaaf-4e95-86f9-9018172c2f7c-cert\") pod \"ingress-canary-8dj77\" (UID: \"1e689b39-eaaf-4e95-86f9-9018172c2f7c\") " pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.690909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.690883 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e689b39-eaaf-4e95-86f9-9018172c2f7c-cert\") pod \"ingress-canary-8dj77\" (UID: \"1e689b39-eaaf-4e95-86f9-9018172c2f7c\") " pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.701062 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.701000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hprz6\" (UniqueName: \"kubernetes.io/projected/1e689b39-eaaf-4e95-86f9-9018172c2f7c-kube-api-access-hprz6\") pod \"ingress-canary-8dj77\" (UID: \"1e689b39-eaaf-4e95-86f9-9018172c2f7c\") " pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.760930 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.760895 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-84w5l" Jul 24 15:37:14.767492 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.767473 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:14.819237 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.819201 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:14.870038 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.870010 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dj77" Jul 24 15:37:14.916762 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.916723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:14.921472 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.921445 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x"] Jul 24 15:37:14.923644 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.923627 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" Jul 24 15:37:14.927668 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.927589 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 24 15:37:14.927668 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.927638 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7fwsw\"" Jul 24 15:37:14.935829 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.935764 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mqhw4"] Jul 24 15:37:14.939999 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.939975 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-84w5l"] Jul 24 15:37:14.941021 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.940903 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x"] Jul 24 15:37:14.941368 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:14.941345 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98cd2a6_0d1a_4be1_9a64_e5331e27c0ae.slice/crio-196588e95052c1ff6e9c2d5015b20ad63aae6750019f9cdb8e575713b7ff2e62 WatchSource:0}: Error finding container 196588e95052c1ff6e9c2d5015b20ad63aae6750019f9cdb8e575713b7ff2e62: Status 404 returned error can't find the container with id 196588e95052c1ff6e9c2d5015b20ad63aae6750019f9cdb8e575713b7ff2e62 Jul 24 15:37:14.943264 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:14.943239 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4eafb17_8bbc_42ec_8284_0330af851b55.slice/crio-e3bb4e033faf10a99a26549153b62a738288760f3a48ab3f509f18ec314c3ee6 WatchSource:0}: Error finding container e3bb4e033faf10a99a26549153b62a738288760f3a48ab3f509f18ec314c3ee6: Status 404 returned error can't find the container with id e3bb4e033faf10a99a26549153b62a738288760f3a48ab3f509f18ec314c3ee6 Jul 24 15:37:14.991449 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:14.991418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4423ad9d-f889-4187-bc2d-85566784ad61-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5s89x\" (UID: \"4423ad9d-f889-4187-bc2d-85566784ad61\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" Jul 24 15:37:15.003083 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.003042 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dj77"] Jul 24 15:37:15.005724 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:15.005695 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e689b39_eaaf_4e95_86f9_9018172c2f7c.slice/crio-e474f96acfd422fd61ec95b1f8da113610c46d1d5eeeb0d6e9a90e8741bc49ef WatchSource:0}: Error finding container e474f96acfd422fd61ec95b1f8da113610c46d1d5eeeb0d6e9a90e8741bc49ef: Status 404 returned error can't find the container with id e474f96acfd422fd61ec95b1f8da113610c46d1d5eeeb0d6e9a90e8741bc49ef Jul 24 15:37:15.007490 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.007391 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:15.092103 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.092064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4423ad9d-f889-4187-bc2d-85566784ad61-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5s89x\" (UID: \"4423ad9d-f889-4187-bc2d-85566784ad61\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" Jul 24 15:37:15.094712 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.094682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4423ad9d-f889-4187-bc2d-85566784ad61-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5s89x\" (UID: \"4423ad9d-f889-4187-bc2d-85566784ad61\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" Jul 24 15:37:15.106858 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.106827 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:15.137972 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.137947 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:15.140511 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.140493 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6g6g\"" Jul 24 15:37:15.140511 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.140500 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 15:37:15.140675 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.140519 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 15:37:15.236078 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.235982 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" Jul 24 15:37:15.304751 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.304689 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:15.391419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.391382 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x"] Jul 24 15:37:15.397348 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:15.397294 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4423ad9d_f889_4187_bc2d_85566784ad61.slice/crio-23accfffc449450c2906f654dab89c3a61621f4e7e761ec450cbe14c390443e3 WatchSource:0}: Error finding container 23accfffc449450c2906f654dab89c3a61621f4e7e761ec450cbe14c390443e3: Status 404 returned error can't find the container with id 23accfffc449450c2906f654dab89c3a61621f4e7e761ec450cbe14c390443e3 Jul 24 15:37:15.398024 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.397988 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84w5l" event={"ID":"c4eafb17-8bbc-42ec-8284-0330af851b55","Type":"ContainerStarted","Data":"e3bb4e033faf10a99a26549153b62a738288760f3a48ab3f509f18ec314c3ee6"} Jul 24 15:37:15.402098 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.402068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dj77" event={"ID":"1e689b39-eaaf-4e95-86f9-9018172c2f7c","Type":"ContainerStarted","Data":"e474f96acfd422fd61ec95b1f8da113610c46d1d5eeeb0d6e9a90e8741bc49ef"} Jul 24 15:37:15.406250 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.406218 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqhw4" event={"ID":"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae","Type":"ContainerStarted","Data":"196588e95052c1ff6e9c2d5015b20ad63aae6750019f9cdb8e575713b7ff2e62"} Jul 24 15:37:15.701257 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.701193 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:15.799490 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:15.799446 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:16.134727 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:16.134688 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:16.134906 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:16.134736 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:16.138754 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:16.138726 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w77kg\"" Jul 24 15:37:16.139691 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:16.139665 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 15:37:16.139969 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:16.139949 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 15:37:16.412046 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:16.411896 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" event={"ID":"4423ad9d-f889-4187-bc2d-85566784ad61","Type":"ContainerStarted","Data":"23accfffc449450c2906f654dab89c3a61621f4e7e761ec450cbe14c390443e3"} Jul 24 15:37:18.328763 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:18.328731 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xwm8x" Jul 24 15:37:18.468678 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:18.468493 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:18.538347 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:18.538292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:18.600645 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:18.600605 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:21.427625 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.427587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" event={"ID":"4423ad9d-f889-4187-bc2d-85566784ad61","Type":"ContainerStarted","Data":"a5b0308385d52fe6f2ac3fc2520fd1c142df70c1ecc0fcb4ee3dcb0d310a28e1"} Jul 24 15:37:21.428079 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.427753 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" Jul 24 15:37:21.429426 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.429402 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dj77" event={"ID":"1e689b39-eaaf-4e95-86f9-9018172c2f7c","Type":"ContainerStarted","Data":"9df57a184e3252a18c278d1529e856aec52abaaa99a2bec2c6484e0773477136"} Jul 24 15:37:21.431845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.431816 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqhw4" event={"ID":"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae","Type":"ContainerStarted","Data":"539dff8e5e7a8e15f5177731b95aa7dc38f17a84295c51140027ef4c1dbd18ad"} Jul 24 15:37:21.431981 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.431870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqhw4" event={"ID":"c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae","Type":"ContainerStarted","Data":"3e62a99b153a6ba85ef913247b31b95ef3ecf6608c3b7a8c3ab3688c029df921"} Jul 24 15:37:21.431981 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.431902 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:21.434395 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.434357 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84w5l" event={"ID":"c4eafb17-8bbc-42ec-8284-0330af851b55","Type":"ContainerStarted","Data":"79cf1c81a99bb029c68b97e4cd0aaa434ef795f423f818af6a2980d3bdba80ee"} Jul 24 15:37:21.434528 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.434488 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" Jul 24 15:37:21.444646 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.444590 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5s89x" podStartSLOduration=2.351283714 podStartE2EDuration="7.444572603s" totalImagesPullingTime="5.093288889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:21.443077358 +0000 UTC m=+64.865678399" watchObservedRunningTime="2026-07-24 15:37:21.444572603 +0000 UTC m=+64.867173632" Jul 24 15:37:21.458451 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.458385 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8dj77" podStartSLOduration=1.640066203 podStartE2EDuration="7.458366796s" totalImagesPullingTime="5.818300593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:21.457424482 +0000 UTC m=+64.880025535" watchObservedRunningTime="2026-07-24 15:37:21.458366796 +0000 UTC m=+64.880967832" Jul 24 15:37:21.475415 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.475162 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mqhw4" podStartSLOduration=1.592475461 podStartE2EDuration="7.475137479s" totalImagesPullingTime="5.882662018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:21.473784706 +0000 UTC m=+64.896385769" watchObservedRunningTime="2026-07-24 15:37:21.475137479 +0000 UTC m=+64.897738509" Jul 24 15:37:21.891845 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.891799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:21.968907 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.968868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:21.971816 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.971786 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ttcjm"] Jul 24 15:37:21.973735 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.973710 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:21.976130 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.976108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 24 15:37:21.976244 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.976187 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2w28j\"" Jul 24 15:37:21.977111 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.977088 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 15:37:21.977486 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.977471 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 15:37:21.977889 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.977753 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 24 15:37:21.977999 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.977919 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 15:37:21.985475 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:21.985447 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ttcjm"] Jul 24 15:37:22.034889 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.034848 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:22.047977 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.047942 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018ea29d-b885-434f-8108-eb5816b0e877-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.048155 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.047981 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df7pf\" (UniqueName: \"kubernetes.io/projected/018ea29d-b885-434f-8108-eb5816b0e877-kube-api-access-df7pf\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.048155 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.048063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018ea29d-b885-434f-8108-eb5816b0e877-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.048155 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.048101 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018ea29d-b885-434f-8108-eb5816b0e877-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.133747 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.133708 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:22.149589 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.149505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018ea29d-b885-434f-8108-eb5816b0e877-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.149589 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.149540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df7pf\" (UniqueName: \"kubernetes.io/projected/018ea29d-b885-434f-8108-eb5816b0e877-kube-api-access-df7pf\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.149782 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.149595 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018ea29d-b885-434f-8108-eb5816b0e877-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.149782 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.149621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018ea29d-b885-434f-8108-eb5816b0e877-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.151331 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.151278 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018ea29d-b885-434f-8108-eb5816b0e877-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.152689 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.152667 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018ea29d-b885-434f-8108-eb5816b0e877-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.152813 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.152790 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018ea29d-b885-434f-8108-eb5816b0e877-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.158871 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.158846 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df7pf\" (UniqueName: \"kubernetes.io/projected/018ea29d-b885-434f-8108-eb5816b0e877-kube-api-access-df7pf\") pod \"prometheus-operator-84d69f794d-ttcjm\" (UID: \"018ea29d-b885-434f-8108-eb5816b0e877\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.237024 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.236944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:22.284834 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.284801 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" Jul 24 15:37:22.429883 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.429804 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-ttcjm"] Jul 24 15:37:22.544337 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.544287 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:22.604229 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.604190 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:22.755580 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.755506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:22.758227 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.758201 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 15:37:22.768890 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.768858 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8239cd3-c74a-4738-9eaf-4b55ce5dc184-metrics-certs\") pod \"network-metrics-daemon-7ffzk\" (UID: \"a8239cd3-c74a-4738-9eaf-4b55ce5dc184\") " pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:22.856266 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.856224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:22.856460 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.856317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:22.860320 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.860273 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 15:37:22.860463 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.860356 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 15:37:22.869834 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.869806 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96502b52-d0fc-49f9-b839-8008f7b76266-original-pull-secret\") pod \"global-pull-secret-syncer-hk86b\" (UID: \"96502b52-d0fc-49f9-b839-8008f7b76266\") " pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:22.870313 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.870281 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 15:37:22.880439 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.880414 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7gnk\" (UniqueName: \"kubernetes.io/projected/549ce8b0-a24a-421b-b074-8391efed6999-kube-api-access-x7gnk\") pod \"network-check-target-xcghk\" (UID: \"549ce8b0-a24a-421b-b074-8391efed6999\") " pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:22.951853 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.951814 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6g6g\"" Jul 24 15:37:22.960108 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:22.960077 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:23.053803 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:23.053772 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w77kg\"" Jul 24 15:37:23.059946 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:23.059914 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hk86b" Jul 24 15:37:23.062056 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:23.062031 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ffzk" Jul 24 15:37:23.710799 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:23.710756 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018ea29d_b885_434f_8108_eb5816b0e877.slice/crio-d350a9ae7213bf281516d1978b4cb97811b92a171106b07064cbc93059c4dc0f WatchSource:0}: Error finding container d350a9ae7213bf281516d1978b4cb97811b92a171106b07064cbc93059c4dc0f: Status 404 returned error can't find the container with id d350a9ae7213bf281516d1978b4cb97811b92a171106b07064cbc93059c4dc0f Jul 24 15:37:24.445959 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:24.445852 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" event={"ID":"018ea29d-b885-434f-8108-eb5816b0e877","Type":"ContainerStarted","Data":"d350a9ae7213bf281516d1978b4cb97811b92a171106b07064cbc93059c4dc0f"} Jul 24 15:37:24.498437 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:24.498369 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hk86b"] Jul 24 15:37:24.504425 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:24.503726 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96502b52_d0fc_49f9_b839_8008f7b76266.slice/crio-b7b83b2b83710eb9c2e028310b3cf01302ccba5d4860c7a739c2454c4cc1eefd WatchSource:0}: Error finding container b7b83b2b83710eb9c2e028310b3cf01302ccba5d4860c7a739c2454c4cc1eefd: Status 404 returned error can't find the container with id b7b83b2b83710eb9c2e028310b3cf01302ccba5d4860c7a739c2454c4cc1eefd Jul 24 15:37:24.508087 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:24.508057 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:24.511265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:24.511241 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xcghk"] Jul 24 15:37:24.514884 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:24.514855 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod549ce8b0_a24a_421b_b074_8391efed6999.slice/crio-e8768cb3bc23e792e6b57c3fdc73db54962f6db985437709ee492396a3639402 WatchSource:0}: Error finding container e8768cb3bc23e792e6b57c3fdc73db54962f6db985437709ee492396a3639402: Status 404 returned error can't find the container with id e8768cb3bc23e792e6b57c3fdc73db54962f6db985437709ee492396a3639402 Jul 24 15:37:24.540483 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:24.540451 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7ffzk"] Jul 24 15:37:24.544340 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:24.544291 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8239cd3_c74a_4738_9eaf_4b55ce5dc184.slice/crio-afac9d56f50d7d0abc217e8c8d230c227ac88c60e3d9e7f83761afad827a29cf WatchSource:0}: Error finding container afac9d56f50d7d0abc217e8c8d230c227ac88c60e3d9e7f83761afad827a29cf: Status 404 returned error can't find the container with id afac9d56f50d7d0abc217e8c8d230c227ac88c60e3d9e7f83761afad827a29cf Jul 24 15:37:24.574099 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:24.574067 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:25.476507 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:25.476392 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hk86b" event={"ID":"96502b52-d0fc-49f9-b839-8008f7b76266","Type":"ContainerStarted","Data":"b7b83b2b83710eb9c2e028310b3cf01302ccba5d4860c7a739c2454c4cc1eefd"} Jul 24 15:37:25.481090 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:25.480270 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" event={"ID":"018ea29d-b885-434f-8108-eb5816b0e877","Type":"ContainerStarted","Data":"13e001a190a77fe291e24e75da7e4b05d81e8e95641a3c5b26d402c7a474d5e3"} Jul 24 15:37:25.483169 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:25.483096 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ffzk" event={"ID":"a8239cd3-c74a-4738-9eaf-4b55ce5dc184","Type":"ContainerStarted","Data":"afac9d56f50d7d0abc217e8c8d230c227ac88c60e3d9e7f83761afad827a29cf"} Jul 24 15:37:25.487297 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:25.487260 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xcghk" event={"ID":"549ce8b0-a24a-421b-b074-8391efed6999","Type":"ContainerStarted","Data":"e8768cb3bc23e792e6b57c3fdc73db54962f6db985437709ee492396a3639402"} Jul 24 15:37:25.492721 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:25.492686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84w5l" event={"ID":"c4eafb17-8bbc-42ec-8284-0330af851b55","Type":"ContainerStarted","Data":"5953eb76dfc96476a3747a4e6d2b8979fc0241633e8af1993f2b924023ec0f18"} Jul 24 15:37:25.499654 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:25.499598 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" podStartSLOduration=2.952760468 podStartE2EDuration="4.499576149s" totalImagesPullingTime="1.546815681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:25.49824858 +0000 UTC m=+68.920849630" watchObservedRunningTime="2026-07-24 15:37:25.499576149 +0000 UTC m=+68.922177178" Jul 24 15:37:25.633003 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:25.632920 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:26.501509 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:26.501461 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-ttcjm" event={"ID":"018ea29d-b885-434f-8108-eb5816b0e877","Type":"ContainerStarted","Data":"32d994b5d264c8406976cf523082db0b7cb0e04f7760e0da6268f722aa08d407"} Jul 24 15:37:27.122620 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.122340 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:27.250370 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.250287 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:27.343646 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.343150 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr"] Jul 24 15:37:27.345271 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.345234 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:27.358173 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.358146 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.360578 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.360078 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr"] Jul 24 15:37:27.361592 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.361549 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 24 15:37:27.361678 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.361602 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bscf2\"" Jul 24 15:37:27.361678 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.361565 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 24 15:37:27.365881 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.365766 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7"] Jul 24 15:37:27.379554 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.379490 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h8642"] Jul 24 15:37:27.379680 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.379612 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.390632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.386782 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 24 15:37:27.390632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.386855 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 24 15:37:27.390632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.387507 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 24 15:37:27.390632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.387785 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7ps46\"" Jul 24 15:37:27.390632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.388434 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7"] Jul 24 15:37:27.390632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.388750 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.395020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.393966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 15:37:27.395020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.394053 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9cbmn\"" Jul 24 15:37:27.395020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.394115 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 15:37:27.396837 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.396817 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 15:37:27.425385 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.425350 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-root\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-sys\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498731 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfg2l\" (UniqueName: \"kubernetes.io/projected/160eac8d-2761-4d6e-82bc-1af9ad1cd529-kube-api-access-cfg2l\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-textfile\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp98w\" (UniqueName: \"kubernetes.io/projected/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-kube-api-access-wp98w\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f47f7cdb-06a5-4319-a54d-4c6cb078e021-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498879 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498906 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f47f7cdb-06a5-4319-a54d-4c6cb078e021-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.498972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r4ln\" (UniqueName: \"kubernetes.io/projected/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-api-access-9r4ln\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.499685 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.499066 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/160eac8d-2761-4d6e-82bc-1af9ad1cd529-metrics-client-ca\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499685 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.499112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-tls\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499685 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.499160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-wtmp\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.499685 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.499186 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.499685 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.499214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.499685 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.499244 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.509859 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.509822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ffzk" event={"ID":"a8239cd3-c74a-4738-9eaf-4b55ce5dc184","Type":"ContainerStarted","Data":"b7b3d79fd0942310b970c5d774c74e491137ed271c57b94c62db3c9b3249253c"} Jul 24 15:37:27.510231 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.509869 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ffzk" event={"ID":"a8239cd3-c74a-4738-9eaf-4b55ce5dc184","Type":"ContainerStarted","Data":"6ad224ef38ca2e3da344df91460d0d885edb91afa0cd0d9eab966385493caded"} Jul 24 15:37:27.521395 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.521357 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:27.528524 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.528468 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7ffzk" podStartSLOduration=68.538605149 podStartE2EDuration="1m10.528446213s" totalImagesPullingTime="1.989841064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:27.5274408 +0000 UTC m=+70.950041860" watchObservedRunningTime="2026-07-24 15:37:27.528446213 +0000 UTC m=+70.951047244" Jul 24 15:37:27.600210 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f47f7cdb-06a5-4319-a54d-4c6cb078e021-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.600210 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.600484 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600282 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r4ln\" (UniqueName: \"kubernetes.io/projected/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-api-access-9r4ln\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.600484 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600360 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/160eac8d-2761-4d6e-82bc-1af9ad1cd529-metrics-client-ca\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.600484 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-tls\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.600484 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-wtmp\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.600484 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.600710 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600499 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.600710 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.600710 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600609 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-root\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.600916 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-sys\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.600916 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600749 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-wtmp\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601077 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-root\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfg2l\" (UniqueName: \"kubernetes.io/projected/160eac8d-2761-4d6e-82bc-1af9ad1cd529-kube-api-access-cfg2l\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601165 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/160eac8d-2761-4d6e-82bc-1af9ad1cd529-sys\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.600751 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f47f7cdb-06a5-4319-a54d-4c6cb078e021-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-textfile\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:27.601378 2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp98w\" (UniqueName: \"kubernetes.io/projected/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-kube-api-access-wp98w\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601393 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:27.601442 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-tls podName:f47f7cdb-06a5-4319-a54d-4c6cb078e021 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:28.101422741 +0000 UTC m=+71.524023759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-kfbt7" (UID: "f47f7cdb-06a5-4319-a54d-4c6cb078e021") : secret "kube-state-metrics-tls" not found Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601438 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/160eac8d-2761-4d6e-82bc-1af9ad1cd529-metrics-client-ca\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f47f7cdb-06a5-4319-a54d-4c6cb078e021-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601523 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601547 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.601555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.602408 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.602408 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.601647 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-textfile\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.602408 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.602266 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-accelerators-collector-config\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.602572 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.602424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.602572 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.602427 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f47f7cdb-06a5-4319-a54d-4c6cb078e021-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.604347 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.604320 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.604715 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.604691 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.605489 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.605468 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.605669 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.605646 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.605815 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.605794 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/160eac8d-2761-4d6e-82bc-1af9ad1cd529-node-exporter-tls\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.613027 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.612991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfg2l\" (UniqueName: \"kubernetes.io/projected/160eac8d-2761-4d6e-82bc-1af9ad1cd529-kube-api-access-cfg2l\") pod \"node-exporter-h8642\" (UID: \"160eac8d-2761-4d6e-82bc-1af9ad1cd529\") " pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.613220 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.613199 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r4ln\" (UniqueName: \"kubernetes.io/projected/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-api-access-9r4ln\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:27.613573 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.613527 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp98w\" (UniqueName: \"kubernetes.io/projected/d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a-kube-api-access-wp98w\") pod \"openshift-state-metrics-b8f49f5c6-s6mpr\" (UID: \"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.622517 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.622467 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:27.674386 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.674281 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" Jul 24 15:37:27.706600 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.706570 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h8642" Jul 24 15:37:27.721934 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.721902 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:27.826035 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:27.825987 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:28.010913 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.010884 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:28.107035 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.107003 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:28.109901 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.109876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f47f7cdb-06a5-4319-a54d-4c6cb078e021-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-kfbt7\" (UID: \"f47f7cdb-06a5-4319-a54d-4c6cb078e021\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:28.300173 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.299874 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" Jul 24 15:37:28.342170 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.342132 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:28.446360 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.446316 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:28.454707 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.454671 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:37:28.459711 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.459689 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.462544 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.462382 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 15:37:28.462544 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.462394 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 15:37:28.462544 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.462406 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 15:37:28.462795 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.462697 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 15:37:28.462911 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.462882 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 15:37:28.463181 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.463151 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 15:37:28.463266 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.463219 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 15:37:28.463576 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.463502 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 15:37:28.463576 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.463521 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 15:37:28.463576 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.463548 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qn5tk\"" Jul 24 15:37:28.473180 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.473156 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:37:28.508731 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.508692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.508928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.508742 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-web-config\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.508928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.508762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.508928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.508844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.508928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.508889 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.508928 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.508926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-out\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.509175 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.508965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b9pz\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-kube-api-access-7b9pz\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.509175 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.509015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.509175 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.509063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.509175 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.509091 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.509175 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.509125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.509175 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.509167 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.509485 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.509205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.610462 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.610368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.610462 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.610408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.610462 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.610439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-out\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611123 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.610474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b9pz\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-kube-api-access-7b9pz\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611123 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.610505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611123 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:37:28.610622 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle podName:5ae215b3-f896-4504-a14b-6fc2b187fc81 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:29.11059383 +0000 UTC m=+72.533194862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81") : configmap references non-existent config key: ca-bundle.crt Jul 24 15:37:28.611123 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611123 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611123 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611471 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611144 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611471 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611181 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611471 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611471 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611277 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-web-config\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611471 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.611789 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.611766 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.612212 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.612182 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.615003 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.614976 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.615114 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.615056 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.615627 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.615602 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.616196 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.616152 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.616504 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.616477 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.617143 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.617123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-web-config\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.617248 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.617223 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-out\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.617449 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.617431 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.617676 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.617657 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:28.624857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:28.624818 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b9pz\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-kube-api-access-7b9pz\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:29.116590 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.116548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:29.117556 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.117528 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:29.374921 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.374828 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:37:29.482912 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.482396 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx"] Jul 24 15:37:29.486057 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.486027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.488458 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.488432 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 24 15:37:29.488458 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.488446 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 24 15:37:29.488672 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.488602 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 24 15:37:29.489581 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.489558 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-535810ofkibdi\"" Jul 24 15:37:29.489581 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.489574 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 24 15:37:29.489762 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.489615 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7gh75\"" Jul 24 15:37:29.489762 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.489558 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 24 15:37:29.497472 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.497444 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx"] Jul 24 15:37:29.621085 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621049 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-grpc-tls\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.621596 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx8jp\" (UniqueName: \"kubernetes.io/projected/ca32c914-381b-4785-9954-30036cbd5537-kube-api-access-wx8jp\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.621596 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.621596 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.621596 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca32c914-381b-4785-9954-30036cbd5537-metrics-client-ca\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.621596 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621409 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.621596 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621444 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.621596 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.621474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-tls\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722350 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722350 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722293 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722350 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-tls\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722647 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-grpc-tls\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722647 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722395 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx8jp\" (UniqueName: \"kubernetes.io/projected/ca32c914-381b-4785-9954-30036cbd5537-kube-api-access-wx8jp\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722647 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722443 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722647 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.722647 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.722521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca32c914-381b-4785-9954-30036cbd5537-metrics-client-ca\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.723545 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.723453 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca32c914-381b-4785-9954-30036cbd5537-metrics-client-ca\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.725728 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.725697 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.726098 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.726066 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.726209 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.726148 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.726209 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.726174 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-tls\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.726348 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.726246 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.726398 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.726362 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca32c914-381b-4785-9954-30036cbd5537-secret-grpc-tls\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.731990 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.731965 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx8jp\" (UniqueName: \"kubernetes.io/projected/ca32c914-381b-4785-9954-30036cbd5537-kube-api-access-wx8jp\") pod \"thanos-querier-7fd8ff8b45-fr9xx\" (UID: \"ca32c914-381b-4785-9954-30036cbd5537\") " pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:29.797960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:29.797919 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:30.282478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:30.282442 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:30.358231 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:30.358194 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:30.424208 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:30.424170 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:30.584160 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:30.584069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:31.439983 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.439952 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mqhw4" Jul 24 15:37:31.671488 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:31.671441 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod160eac8d_2761_4d6e_82bc_1af9ad1cd529.slice/crio-15fe9f0d1aec1acacb3d915f45afd7c52fcbae38c6a4cb06beb8459b73bdd31b WatchSource:0}: Error finding container 15fe9f0d1aec1acacb3d915f45afd7c52fcbae38c6a4cb06beb8459b73bdd31b: Status 404 returned error can't find the container with id 15fe9f0d1aec1acacb3d915f45afd7c52fcbae38c6a4cb06beb8459b73bdd31b Jul 24 15:37:31.749830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.749798 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79bf54789c-s2xzw"] Jul 24 15:37:31.769640 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.767866 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79bf54789c-s2xzw"] Jul 24 15:37:31.769640 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.768032 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.776750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.772928 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 24 15:37:31.776750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.773011 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 24 15:37:31.776750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.773156 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qgtwk\"" Jul 24 15:37:31.776750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.774023 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-55ifgj9fepp24\"" Jul 24 15:37:31.776750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.774292 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 24 15:37:31.776750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.774565 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 24 15:37:31.846587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.846138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-secret-metrics-server-client-certs\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.846587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.846208 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d2dd966b-ad1e-4638-88b5-c002a015d6b4-audit-log\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.846587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.846257 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94642\" (UniqueName: \"kubernetes.io/projected/d2dd966b-ad1e-4638-88b5-c002a015d6b4-kube-api-access-94642\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.846587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.846295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-secret-metrics-server-tls\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.846587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.846371 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d2dd966b-ad1e-4638-88b5-c002a015d6b4-metrics-server-audit-profiles\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.846587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.846401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2dd966b-ad1e-4638-88b5-c002a015d6b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.846587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.846436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-client-ca-bundle\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.903629 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.903595 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr"] Jul 24 15:37:31.911749 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:31.911698 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08a2e9c_a4de_46ce_b7b7_56f20a1ce80a.slice/crio-70bebd27669e0d7460ba5df5d3a0ca5e652a3db8a7560bd456ba7eb71289793b WatchSource:0}: Error finding container 70bebd27669e0d7460ba5df5d3a0ca5e652a3db8a7560bd456ba7eb71289793b: Status 404 returned error can't find the container with id 70bebd27669e0d7460ba5df5d3a0ca5e652a3db8a7560bd456ba7eb71289793b Jul 24 15:37:31.922054 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.921980 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx"] Jul 24 15:37:31.927367 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:31.927337 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca32c914_381b_4785_9954_30036cbd5537.slice/crio-2221d145e87a1b424a170e8748a0611dbbdcbf994b649262c9737102cf3d559c WatchSource:0}: Error finding container 2221d145e87a1b424a170e8748a0611dbbdcbf994b649262c9737102cf3d559c: Status 404 returned error can't find the container with id 2221d145e87a1b424a170e8748a0611dbbdcbf994b649262c9737102cf3d559c Jul 24 15:37:31.946067 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.945988 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:37:31.947378 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.947345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-secret-metrics-server-tls\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.947525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.947404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d2dd966b-ad1e-4638-88b5-c002a015d6b4-metrics-server-audit-profiles\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.947525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.947442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2dd966b-ad1e-4638-88b5-c002a015d6b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.947525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.947476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-client-ca-bundle\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.947525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.947502 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-secret-metrics-server-client-certs\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.947737 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.947551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d2dd966b-ad1e-4638-88b5-c002a015d6b4-audit-log\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.947737 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.947604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94642\" (UniqueName: \"kubernetes.io/projected/d2dd966b-ad1e-4638-88b5-c002a015d6b4-kube-api-access-94642\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.948490 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.948154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2dd966b-ad1e-4638-88b5-c002a015d6b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.948490 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.948434 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d2dd966b-ad1e-4638-88b5-c002a015d6b4-audit-log\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.948648 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.948545 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d2dd966b-ad1e-4638-88b5-c002a015d6b4-metrics-server-audit-profiles\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.950930 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.950909 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-secret-metrics-server-tls\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.951469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.951421 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-secret-metrics-server-client-certs\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.951809 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.951784 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2dd966b-ad1e-4638-88b5-c002a015d6b4-client-ca-bundle\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.952211 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:31.952185 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae215b3_f896_4504_a14b_6fc2b187fc81.slice/crio-e141879402fdb8cf0ad4fa0baadd53860445364c54419949ff3aa474bd5a6d65 WatchSource:0}: Error finding container e141879402fdb8cf0ad4fa0baadd53860445364c54419949ff3aa474bd5a6d65: Status 404 returned error can't find the container with id e141879402fdb8cf0ad4fa0baadd53860445364c54419949ff3aa474bd5a6d65 Jul 24 15:37:31.956432 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.956408 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94642\" (UniqueName: \"kubernetes.io/projected/d2dd966b-ad1e-4638-88b5-c002a015d6b4-kube-api-access-94642\") pod \"metrics-server-79bf54789c-s2xzw\" (UID: \"d2dd966b-ad1e-4638-88b5-c002a015d6b4\") " pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:31.962211 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:31.962183 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7"] Jul 24 15:37:31.964350 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:31.964287 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47f7cdb_06a5_4319_a54d_4c6cb078e021.slice/crio-61ff37c955cbb4ecf0405b084de2d97c16ecb60501ce972e371f56ebc0b2988f WatchSource:0}: Error finding container 61ff37c955cbb4ecf0405b084de2d97c16ecb60501ce972e371f56ebc0b2988f: Status 404 returned error can't find the container with id 61ff37c955cbb4ecf0405b084de2d97c16ecb60501ce972e371f56ebc0b2988f Jul 24 15:37:32.119773 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.119738 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:32.127373 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.127341 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz"] Jul 24 15:37:32.131911 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.131890 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.133139 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.133108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:32.134371 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.134348 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 24 15:37:32.134629 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.134464 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wl4q6\"" Jul 24 15:37:32.142061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.142033 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz"] Jul 24 15:37:32.246349 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.246317 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:32.250459 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.250428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44a77978-b18b-439a-88f1-f26c809e0eaf-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bm9vz\" (UID: \"44a77978-b18b-439a-88f1-f26c809e0eaf\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.250630 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.250476 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j88bd\" (UniqueName: \"kubernetes.io/projected/44a77978-b18b-439a-88f1-f26c809e0eaf-kube-api-access-j88bd\") pod \"monitoring-plugin-68b7fc9574-bm9vz\" (UID: \"44a77978-b18b-439a-88f1-f26c809e0eaf\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.279872 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.279830 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79bf54789c-s2xzw"] Jul 24 15:37:32.283479 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:32.283447 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2dd966b_ad1e_4638_88b5_c002a015d6b4.slice/crio-f4202f7cfd5f7176c3b6e7c24a6e3cdc454b7179daaac2d9fa6cd6f37b22b0c0 WatchSource:0}: Error finding container f4202f7cfd5f7176c3b6e7c24a6e3cdc454b7179daaac2d9fa6cd6f37b22b0c0: Status 404 returned error can't find the container with id f4202f7cfd5f7176c3b6e7c24a6e3cdc454b7179daaac2d9fa6cd6f37b22b0c0 Jul 24 15:37:32.351253 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.351217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j88bd\" (UniqueName: \"kubernetes.io/projected/44a77978-b18b-439a-88f1-f26c809e0eaf-kube-api-access-j88bd\") pod \"monitoring-plugin-68b7fc9574-bm9vz\" (UID: \"44a77978-b18b-439a-88f1-f26c809e0eaf\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.351493 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.351392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44a77978-b18b-439a-88f1-f26c809e0eaf-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bm9vz\" (UID: \"44a77978-b18b-439a-88f1-f26c809e0eaf\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.353999 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.353978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44a77978-b18b-439a-88f1-f26c809e0eaf-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-bm9vz\" (UID: \"44a77978-b18b-439a-88f1-f26c809e0eaf\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.359816 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.359791 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j88bd\" (UniqueName: \"kubernetes.io/projected/44a77978-b18b-439a-88f1-f26c809e0eaf-kube-api-access-j88bd\") pod \"monitoring-plugin-68b7fc9574-bm9vz\" (UID: \"44a77978-b18b-439a-88f1-f26c809e0eaf\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.444245 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.444201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:32.532785 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.532746 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xcghk" event={"ID":"549ce8b0-a24a-421b-b074-8391efed6999","Type":"ContainerStarted","Data":"522270016ac7c43e4f8fb9915803dc02148eb04ca7963a0d7f3403b539cb9bf6"} Jul 24 15:37:32.532952 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.532849 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:37:32.534050 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.534029 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerStarted","Data":"e141879402fdb8cf0ad4fa0baadd53860445364c54419949ff3aa474bd5a6d65"} Jul 24 15:37:32.535899 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.535875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" event={"ID":"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a","Type":"ContainerStarted","Data":"91037730c4ba8317d6cd89ff37d2bd767903c0524b382751c1cecae39e072909"} Jul 24 15:37:32.536005 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.535919 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" event={"ID":"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a","Type":"ContainerStarted","Data":"67b920000078bfbc52658c6eca2ea7a034e594f9faecf37d95023fa588a102a0"} Jul 24 15:37:32.536005 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.535938 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" event={"ID":"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a","Type":"ContainerStarted","Data":"70bebd27669e0d7460ba5df5d3a0ca5e652a3db8a7560bd456ba7eb71289793b"} Jul 24 15:37:32.537952 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.537920 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84w5l" event={"ID":"c4eafb17-8bbc-42ec-8284-0330af851b55","Type":"ContainerStarted","Data":"0867d8192c9fb82608828aef0e17ccec4b24587d64549ecaa27ca55ac71b9f77"} Jul 24 15:37:32.539061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.539039 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" event={"ID":"ca32c914-381b-4785-9954-30036cbd5537","Type":"ContainerStarted","Data":"2221d145e87a1b424a170e8748a0611dbbdcbf994b649262c9737102cf3d559c"} Jul 24 15:37:32.539973 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.539953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" event={"ID":"f47f7cdb-06a5-4319-a54d-4c6cb078e021","Type":"ContainerStarted","Data":"61ff37c955cbb4ecf0405b084de2d97c16ecb60501ce972e371f56ebc0b2988f"} Jul 24 15:37:32.541359 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.541339 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8642" event={"ID":"160eac8d-2761-4d6e-82bc-1af9ad1cd529","Type":"ContainerStarted","Data":"15fe9f0d1aec1acacb3d915f45afd7c52fcbae38c6a4cb06beb8459b73bdd31b"} Jul 24 15:37:32.542834 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.542810 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hk86b" event={"ID":"96502b52-d0fc-49f9-b839-8008f7b76266","Type":"ContainerStarted","Data":"286d352d098a882217ea458886b6eb850a35c0b6dc210fc755bc46c5f561d479"} Jul 24 15:37:32.543911 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.543889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" event={"ID":"d2dd966b-ad1e-4638-88b5-c002a015d6b4","Type":"ContainerStarted","Data":"f4202f7cfd5f7176c3b6e7c24a6e3cdc454b7179daaac2d9fa6cd6f37b22b0c0"} Jul 24 15:37:32.549775 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.549716 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xcghk" podStartSLOduration=68.392959591 podStartE2EDuration="1m15.549702079s" totalImagesPullingTime="7.156742488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:32.54910796 +0000 UTC m=+75.971709011" watchObservedRunningTime="2026-07-24 15:37:32.549702079 +0000 UTC m=+75.972303108" Jul 24 15:37:32.556797 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.556768 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6787bc55d6-9scqd"] Jul 24 15:37:32.560583 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.560561 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.563061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.563033 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 24 15:37:32.563934 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.563913 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 24 15:37:32.564049 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.564003 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 24 15:37:32.564109 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.564002 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 24 15:37:32.564634 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.564617 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-27glq\"" Jul 24 15:37:32.565713 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.565665 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 24 15:37:32.572115 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.572081 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 24 15:37:32.573027 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.572975 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hk86b" podStartSLOduration=68.400684885 podStartE2EDuration="1m15.572956172s" totalImagesPullingTime="7.172271287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:32.572751465 +0000 UTC m=+75.995352507" watchObservedRunningTime="2026-07-24 15:37:32.572956172 +0000 UTC m=+75.995557220" Jul 24 15:37:32.580802 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.580683 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6787bc55d6-9scqd"] Jul 24 15:37:32.607886 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.607840 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-84w5l" podStartSLOduration=2.058493248 podStartE2EDuration="18.607823486s" totalImagesPullingTime="16.549330238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:37:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:32.606683469 +0000 UTC m=+76.029284518" watchObservedRunningTime="2026-07-24 15:37:32.607823486 +0000 UTC m=+76.030424524" Jul 24 15:37:32.654626 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.654596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-serving-certs-ca-bundle\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.654752 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.654659 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-federate-client-tls\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.654752 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.654686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-metrics-client-ca\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.654752 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.654706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-secret-telemeter-client\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.654932 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.654904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-telemeter-client-tls\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.655002 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.654985 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87sf\" (UniqueName: \"kubernetes.io/projected/617fa295-bdfe-4dd1-a0ef-d161445e7092-kube-api-access-t87sf\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.655056 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.655020 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.655110 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.655069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.690268 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.690240 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz"] Jul 24 15:37:32.693404 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:32.693370 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a77978_b18b_439a_88f1_f26c809e0eaf.slice/crio-7f024c5c7495918457faf62208d9d0b4e3764d0be8672f20809b829203fd58f5 WatchSource:0}: Error finding container 7f024c5c7495918457faf62208d9d0b4e3764d0be8672f20809b829203fd58f5: Status 404 returned error can't find the container with id 7f024c5c7495918457faf62208d9d0b4e3764d0be8672f20809b829203fd58f5 Jul 24 15:37:32.756000 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.755974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-federate-client-tls\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756107 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756054 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-metrics-client-ca\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756107 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756098 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-secret-telemeter-client\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756213 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-telemeter-client-tls\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756264 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t87sf\" (UniqueName: \"kubernetes.io/projected/617fa295-bdfe-4dd1-a0ef-d161445e7092-kube-api-access-t87sf\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756264 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756326 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-serving-certs-ca-bundle\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.756940 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756912 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-metrics-client-ca\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.757045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.756952 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-serving-certs-ca-bundle\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.758452 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.758397 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/617fa295-bdfe-4dd1-a0ef-d161445e7092-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.760067 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.760019 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-federate-client-tls\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.761520 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.761471 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.762959 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.762826 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-telemeter-client-tls\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.764479 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.764436 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/617fa295-bdfe-4dd1-a0ef-d161445e7092-secret-telemeter-client\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.768907 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.768852 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87sf\" (UniqueName: \"kubernetes.io/projected/617fa295-bdfe-4dd1-a0ef-d161445e7092-kube-api-access-t87sf\") pod \"telemeter-client-6787bc55d6-9scqd\" (UID: \"617fa295-bdfe-4dd1-a0ef-d161445e7092\") " pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:32.773131 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.773102 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:32.873543 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.873464 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:32.875776 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:32.875532 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" Jul 24 15:37:33.069113 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.068997 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6787bc55d6-9scqd"] Jul 24 15:37:33.470633 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:33.470588 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617fa295_bdfe_4dd1_a0ef_d161445e7092.slice/crio-a168d30f1936f3185ab46f8bfc5ab3628e36eb2233f8813cfeccd34bffdeaba4 WatchSource:0}: Error finding container a168d30f1936f3185ab46f8bfc5ab3628e36eb2233f8813cfeccd34bffdeaba4: Status 404 returned error can't find the container with id a168d30f1936f3185ab46f8bfc5ab3628e36eb2233f8813cfeccd34bffdeaba4 Jul 24 15:37:33.553068 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.553026 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" event={"ID":"617fa295-bdfe-4dd1-a0ef-d161445e7092","Type":"ContainerStarted","Data":"a168d30f1936f3185ab46f8bfc5ab3628e36eb2233f8813cfeccd34bffdeaba4"} Jul 24 15:37:33.555053 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.554953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" event={"ID":"44a77978-b18b-439a-88f1-f26c809e0eaf","Type":"ContainerStarted","Data":"7f024c5c7495918457faf62208d9d0b4e3764d0be8672f20809b829203fd58f5"} Jul 24 15:37:33.559192 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.557712 2560 generic.go:356] "Generic (PLEG): container finished" podID="160eac8d-2761-4d6e-82bc-1af9ad1cd529" containerID="eccb3f2aeacb574033d32a55eb9b614161f85d8e86d89ef17e5be7c7751b122e" exitCode=0 Jul 24 15:37:33.559192 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.557914 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8642" event={"ID":"160eac8d-2761-4d6e-82bc-1af9ad1cd529","Type":"ContainerDied","Data":"eccb3f2aeacb574033d32a55eb9b614161f85d8e86d89ef17e5be7c7751b122e"} Jul 24 15:37:33.657094 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.657055 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:37:33.667395 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.667362 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:33.670927 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.670899 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.674032 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.674009 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 15:37:33.674198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.674008 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 15:37:33.674274 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.674230 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 15:37:33.674504 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.674480 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bfmvn529bl84h\"" Jul 24 15:37:33.675218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.674748 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 15:37:33.675218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.674887 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 15:37:33.675218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.675005 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 15:37:33.677332 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.675650 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 15:37:33.677332 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.675856 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 15:37:33.677332 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.676046 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mrd66\"" Jul 24 15:37:33.677332 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.676227 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 15:37:33.677999 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.677794 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 15:37:33.677999 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.677838 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 15:37:33.677999 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.677796 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 15:37:33.686696 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.686256 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 15:37:33.686696 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.686529 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:37:33.689890 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.689866 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 15:37:33.768440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.768440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768238 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.768440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768277 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.768440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768320 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.768440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.768440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768532 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-config\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768650 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp2dm\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-kube-api-access-wp2dm\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768860 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-web-config\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768886 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-config-out\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768937 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.769022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.768962 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-web-config\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870495 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-config-out\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870614 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870777 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870869 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-config\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.870970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.872865 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.871001 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp2dm\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-kube-api-access-wp2dm\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.874127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.871030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.874127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.871053 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.874127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.871074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.874127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.873021 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.874127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.873295 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.874127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.873563 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.874127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.873951 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.875650 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.875614 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.878163 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.878053 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.878984 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.878663 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.878984 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.878713 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-config-out\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.878984 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.878760 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.878984 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.878956 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-web-config\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.878984 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.878956 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.879319 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.879043 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.879384 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.879331 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.879702 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.879676 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.879872 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.879828 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.881794 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.881753 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.882045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.882018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.882804 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.882783 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.885156 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.885134 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp2dm\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-kube-api-access-wp2dm\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.889145 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.889098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-config\") pod \"prometheus-k8s-0\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:33.993905 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:33.993496 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:34.457736 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:34.457696 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:34.558135 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:34.558040 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:34.613171 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:34.613132 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:34.742009 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:34.741921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:34.843394 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:34.843359 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:34.943342 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:34.943271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:35.544754 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:35.544719 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:35.752226 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:35.752153 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:35.935806 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:35.935773 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:35.994540 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:35.994503 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:36.138102 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.138068 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:36.417746 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.415264 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:37:36.420484 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:37:36.420206 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode049b69d_59a7_4279_ae6e_f324b59e09bb.slice/crio-2ae3a4561a9279fc26476486e92026964e29b3efcf70847ad1e86406bcc7c5cd WatchSource:0}: Error finding container 2ae3a4561a9279fc26476486e92026964e29b3efcf70847ad1e86406bcc7c5cd: Status 404 returned error can't find the container with id 2ae3a4561a9279fc26476486e92026964e29b3efcf70847ad1e86406bcc7c5cd Jul 24 15:37:36.446412 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.446117 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:36.548875 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.548406 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:36.586967 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.581214 2560 generic.go:356] "Generic (PLEG): container finished" podID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerID="13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926" exitCode=0 Jul 24 15:37:36.586967 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.581326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926"} Jul 24 15:37:36.611122 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.611011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" event={"ID":"d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a","Type":"ContainerStarted","Data":"cbe01091ac5ad4aaceaf84bb17c743a918b4dfa6fd75b69531247b2e7c87ca32"} Jul 24 15:37:36.627333 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.626055 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerStarted","Data":"2ae3a4561a9279fc26476486e92026964e29b3efcf70847ad1e86406bcc7c5cd"} Jul 24 15:37:36.634744 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.631280 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" event={"ID":"ca32c914-381b-4785-9954-30036cbd5537","Type":"ContainerStarted","Data":"1f551598f0e7b17d49095fcb8049c973f570d75ec764182b11b6211bf3cb4f2d"} Jul 24 15:37:36.634744 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.631415 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" event={"ID":"ca32c914-381b-4785-9954-30036cbd5537","Type":"ContainerStarted","Data":"286aeac3fe7fe680e48f4c132800256a7a4a48e90a585013f715e1321f1db6a9"} Jul 24 15:37:36.634744 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.634058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:36.638680 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.638627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" event={"ID":"f47f7cdb-06a5-4319-a54d-4c6cb078e021","Type":"ContainerStarted","Data":"5950ef2f944dbada9c5d3afc59da7cbe3ddc66b86df6fd7b2bfc631a3629079a"} Jul 24 15:37:36.638786 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.638696 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" event={"ID":"f47f7cdb-06a5-4319-a54d-4c6cb078e021","Type":"ContainerStarted","Data":"81fc92ec0646d9b7a833d006dce5ae73622129cec34ec63598d8ac09cbeae3e8"} Jul 24 15:37:36.642733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.642669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8642" event={"ID":"160eac8d-2761-4d6e-82bc-1af9ad1cd529","Type":"ContainerStarted","Data":"ee50b8f997af69032b6df9750ac8abd18dfb4c9f45da3387aa0d6abbfe7b886a"} Jul 24 15:37:36.642733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.642726 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h8642" event={"ID":"160eac8d-2761-4d6e-82bc-1af9ad1cd529","Type":"ContainerStarted","Data":"660fb9876dfcbdf5e5025e039d9074cfddac7e9a0a7c5c985a753fa25548fcb1"} Jul 24 15:37:36.645268 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.645188 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" event={"ID":"617fa295-bdfe-4dd1-a0ef-d161445e7092","Type":"ContainerStarted","Data":"2eede1a893dcd55846de65b47926d32303932010a27f997d0f2aeb8f49e44dc9"} Jul 24 15:37:36.646884 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.646829 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-s6mpr" podStartSLOduration=5.932453311 podStartE2EDuration="9.646810972s" totalImagesPullingTime="3.714357661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:36.646039964 +0000 UTC m=+80.068641019" watchObservedRunningTime="2026-07-24 15:37:36.646810972 +0000 UTC m=+80.069412002" Jul 24 15:37:36.649170 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.649137 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" event={"ID":"44a77978-b18b-439a-88f1-f26c809e0eaf","Type":"ContainerStarted","Data":"edc944a2a5da5a0e70b98632cd769776b8cf0c7de70aaa802b72f0ad93c66f5f"} Jul 24 15:37:36.649964 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.649944 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:36.652672 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.652644 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" event={"ID":"d2dd966b-ad1e-4638-88b5-c002a015d6b4","Type":"ContainerStarted","Data":"ced72d5896d25e05979555976323a71bbcfdfc83b17a4570d2946e7ec44452d9"} Jul 24 15:37:36.655834 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.655814 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" Jul 24 15:37:36.668044 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.667627 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h8642" podStartSLOduration=8.65531413 podStartE2EDuration="9.667608368s" totalImagesPullingTime="920.924637ms" totalInitContainerRuntime="91.369601ms" isStatefulPod=true podCreationTimestamp="2026-07-24 15:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:36.667237423 +0000 UTC m=+80.089838470" watchObservedRunningTime="2026-07-24 15:37:36.667608368 +0000 UTC m=+80.090209397" Jul 24 15:37:36.723495 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.723448 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-bm9vz" podStartSLOduration=1.1580931269999999 podStartE2EDuration="4.723431006s" totalImagesPullingTime="3.565337879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:36.722413436 +0000 UTC m=+80.145014491" watchObservedRunningTime="2026-07-24 15:37:36.723431006 +0000 UTC m=+80.146032029" Jul 24 15:37:36.724516 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.724495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:36.747437 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:36.747380 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" podStartSLOduration=1.779163754 podStartE2EDuration="5.74735353s" totalImagesPullingTime="3.968189776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:36.74379834 +0000 UTC m=+80.166399385" watchObservedRunningTime="2026-07-24 15:37:36.74735353 +0000 UTC m=+80.169954562" Jul 24 15:37:37.664388 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.661550 2560 generic.go:356] "Generic (PLEG): container finished" podID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerID="eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8" exitCode=0 Jul 24 15:37:37.664388 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.661660 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8"} Jul 24 15:37:37.669245 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.669216 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" event={"ID":"ca32c914-381b-4785-9954-30036cbd5537","Type":"ContainerStarted","Data":"768661c670ec70a9b6f518e0b4888034d79a75386b915e01b661754feff23c5a"} Jul 24 15:37:37.672917 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.672841 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" event={"ID":"f47f7cdb-06a5-4319-a54d-4c6cb078e021","Type":"ContainerStarted","Data":"fe19a57ce7c3a9a0031d63b53fc489856db1f05f12255a3f606781680a14f3a4"} Jul 24 15:37:37.678374 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.677810 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" event={"ID":"617fa295-bdfe-4dd1-a0ef-d161445e7092","Type":"ContainerStarted","Data":"bdc006eb40d7fc9aca763ee1d37485d7b62c2f75d8b754a804d4faf7198361ce"} Jul 24 15:37:37.678374 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.677885 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" event={"ID":"617fa295-bdfe-4dd1-a0ef-d161445e7092","Type":"ContainerStarted","Data":"438734a7d2abdcfa2bc64ca4241ccd2b2fa6e10f72c66b3ac3e40f1618b17472"} Jul 24 15:37:37.707193 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.707161 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:37.711606 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.711562 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-kfbt7" podStartSLOduration=6.475277311 podStartE2EDuration="10.711545517s" totalImagesPullingTime="4.236268206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:37.71001617 +0000 UTC m=+81.132617196" watchObservedRunningTime="2026-07-24 15:37:37.711545517 +0000 UTC m=+81.134146590" Jul 24 15:37:37.735500 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.735446 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6787bc55d6-9scqd" podStartSLOduration=2.9416597810000003 podStartE2EDuration="5.735425831s" totalImagesPullingTime="2.79376605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:37.73383693 +0000 UTC m=+81.156437984" watchObservedRunningTime="2026-07-24 15:37:37.735425831 +0000 UTC m=+81.158026898" Jul 24 15:37:37.808836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.808801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:37.968469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:37.968369 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:38.063340 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.063296 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:38.686656 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.686610 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerStarted","Data":"326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53"} Jul 24 15:37:38.687089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.686743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerStarted","Data":"3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962"} Jul 24 15:37:38.687089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.686787 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerStarted","Data":"7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169"} Jul 24 15:37:38.687089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.686829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerStarted","Data":"ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1"} Jul 24 15:37:38.687089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.686871 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerStarted","Data":"04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d"} Jul 24 15:37:38.687089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.686906 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerStarted","Data":"2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7"} Jul 24 15:37:38.691473 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.691433 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" event={"ID":"ca32c914-381b-4785-9954-30036cbd5537","Type":"ContainerStarted","Data":"9ea5775881533aaee55abce16ea0600efe44163fec83eea829eaeb4919ce364e"} Jul 24 15:37:38.691632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.691507 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" event={"ID":"ca32c914-381b-4785-9954-30036cbd5537","Type":"ContainerStarted","Data":"95a9c922ba3671c692fec33f321e387433dc686a732401309ee8b00e0888fc2b"} Jul 24 15:37:38.691632 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.691538 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" event={"ID":"ca32c914-381b-4785-9954-30036cbd5537","Type":"ContainerStarted","Data":"974f4181d7c1a352b3f7a2f7f1318dd7a541e03f361c5b981ead938187d62cdf"} Jul 24 15:37:38.723786 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.723715 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.997062823 podStartE2EDuration="10.72369008s" totalImagesPullingTime="5.691628949s" totalInitContainerRuntime="34.998308ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:38.718761002 +0000 UTC m=+82.141362056" watchObservedRunningTime="2026-07-24 15:37:38.72369008 +0000 UTC m=+82.146291110" Jul 24 15:37:38.748169 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:38.748072 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" podStartSLOduration=4.596104506 podStartE2EDuration="9.748054512s" totalImagesPullingTime="5.151950006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:38.746195027 +0000 UTC m=+82.168796077" watchObservedRunningTime="2026-07-24 15:37:38.748054512 +0000 UTC m=+82.170655541" Jul 24 15:37:39.698651 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:39.698544 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:40.747969 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:40.747921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:40.849065 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:40.849021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:41.671094 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.671013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:41.710679 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.710645 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerStarted","Data":"f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce"} Jul 24 15:37:41.710839 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.710708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerStarted","Data":"dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0"} Jul 24 15:37:41.710839 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.710741 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerStarted","Data":"281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3"} Jul 24 15:37:41.710839 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.710769 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerStarted","Data":"a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f"} Jul 24 15:37:41.710839 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.710802 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerStarted","Data":"e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c"} Jul 24 15:37:41.710839 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.710826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerStarted","Data":"782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8"} Jul 24 15:37:41.752017 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.751966 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.252626811 podStartE2EDuration="8.751946917s" totalImagesPullingTime="3.437924712s" totalInitContainerRuntime="61.395394ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:41.750097826 +0000 UTC m=+85.172698883" watchObservedRunningTime="2026-07-24 15:37:41.751946917 +0000 UTC m=+85.174547994" Jul 24 15:37:41.758891 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:41.758863 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:42.162750 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:42.162712 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:42.263781 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:42.263747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:43.776057 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:43.776019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:43.876431 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:43.876399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:43.994960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:43.994920 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:37:45.709056 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:45.709028 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7fd8ff8b45-fr9xx" Jul 24 15:37:46.791706 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:46.791672 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:46.857639 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:46.857585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:50.152897 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:50.152860 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:52.120731 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:52.120703 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:52.121139 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:52.120758 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:37:53.953473 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:53.953440 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:54.009420 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:54.009368 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:56.681246 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:56.681202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:56.745733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:56.745701 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:58.966410 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:58.966371 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:37:59.029880 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:37:59.029836 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:01.235182 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:01.235144 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:03.561248 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:03.561213 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xcghk" Jul 24 15:38:11.684927 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:11.684895 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:11.786815 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:11.786786 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:12.126101 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:12.126073 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:38:12.130005 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:12.129974 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79bf54789c-s2xzw" Jul 24 15:38:24.384279 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:24.384242 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:33.995078 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:33.995038 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:34.016812 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:34.016782 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:34.948198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:34.948169 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:43.543675 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:43.543640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:43.629830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:43.629791 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:47.853524 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:47.853484 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:38:47.985194 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:47.985133 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="alertmanager" containerID="cri-o://2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7" gracePeriod=120 Jul 24 15:38:47.985417 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:47.985216 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-metric" containerID="cri-o://3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962" gracePeriod=120 Jul 24 15:38:47.985417 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:47.985248 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="config-reloader" containerID="cri-o://04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d" gracePeriod=120 Jul 24 15:38:47.985417 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:47.985270 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy" containerID="cri-o://7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169" gracePeriod=120 Jul 24 15:38:47.985417 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:47.985259 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-web" containerID="cri-o://ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1" gracePeriod=120 Jul 24 15:38:47.985417 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:47.985226 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="prom-label-proxy" containerID="cri-o://326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53" gracePeriod=120 Jul 24 15:38:48.992372 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992334 2560 generic.go:356] "Generic (PLEG): container finished" podID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerID="326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53" exitCode=0 Jul 24 15:38:48.992372 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992371 2560 generic.go:356] "Generic (PLEG): container finished" podID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerID="3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962" exitCode=0 Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992381 2560 generic.go:356] "Generic (PLEG): container finished" podID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerID="7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169" exitCode=0 Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992388 2560 generic.go:356] "Generic (PLEG): container finished" podID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerID="04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d" exitCode=0 Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992393 2560 generic.go:356] "Generic (PLEG): container finished" podID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerID="2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7" exitCode=0 Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992336 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53"} Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992522 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962"} Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169"} Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992577 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d"} Jul 24 15:38:48.992788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:48.992602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7"} Jul 24 15:38:49.249502 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.249445 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:49.356896 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.356860 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-metrics-client-ca\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357086 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.356907 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357086 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.356933 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357086 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.356964 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-web\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357086 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357022 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-web-config\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357086 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357047 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-out\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357086 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357073 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357107 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-main-tls\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357164 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-main-db\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357239 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-volume\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357272 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-tls-assets\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357347 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-cluster-tls-config\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357353 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:49.357419 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357375 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b9pz\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-kube-api-access-7b9pz\") pod \"5ae215b3-f896-4504-a14b-6fc2b187fc81\" (UID: \"5ae215b3-f896-4504-a14b-6fc2b187fc81\") " Jul 24 15:38:49.357734 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357568 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:38:49.357734 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357667 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-main-db\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.357734 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357685 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-metrics-client-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.357898 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.357784 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:49.361181 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.361060 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:49.361362 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.361258 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:49.361524 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.361487 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:49.361524 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.361503 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-out" (OuterVolumeSpecName: "config-out") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:38:49.361658 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.361509 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-kube-api-access-7b9pz" (OuterVolumeSpecName: "kube-api-access-7b9pz") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "kube-api-access-7b9pz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:38:49.361703 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.361653 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:49.361827 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.361802 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-volume" (OuterVolumeSpecName: "config-volume") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:49.362134 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.362113 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:38:49.365844 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.365821 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:49.372509 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.372478 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-web-config" (OuterVolumeSpecName: "web-config") pod "5ae215b3-f896-4504-a14b-6fc2b187fc81" (UID: "5ae215b3-f896-4504-a14b-6fc2b187fc81"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:49.458846 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458818 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-cluster-tls-config\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.458846 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458842 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b9pz\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-kube-api-access-7b9pz\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.458846 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458853 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458863 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458872 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458884 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-web-config\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458891 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-out\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458900 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ae215b3-f896-4504-a14b-6fc2b187fc81-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458911 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-secret-alertmanager-main-tls\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458918 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ae215b3-f896-4504-a14b-6fc2b187fc81-config-volume\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:49.459076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:49.458926 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ae215b3-f896-4504-a14b-6fc2b187fc81-tls-assets\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:50.000208 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.000173 2560 generic.go:356] "Generic (PLEG): container finished" podID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerID="ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1" exitCode=0 Jul 24 15:38:50.000719 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.000251 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.000719 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.000249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1"} Jul 24 15:38:50.000719 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.000401 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ae215b3-f896-4504-a14b-6fc2b187fc81","Type":"ContainerDied","Data":"e141879402fdb8cf0ad4fa0baadd53860445364c54419949ff3aa474bd5a6d65"} Jul 24 15:38:50.000719 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.000459 2560 scope.go:117] "RemoveContainer" containerID="326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53" Jul 24 15:38:50.008734 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.008709 2560 scope.go:117] "RemoveContainer" containerID="3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962" Jul 24 15:38:50.016478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.016450 2560 scope.go:117] "RemoveContainer" containerID="7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169" Jul 24 15:38:50.023849 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.023832 2560 scope.go:117] "RemoveContainer" containerID="ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1" Jul 24 15:38:50.029096 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.029052 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:38:50.030769 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.030745 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:38:50.033418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.033403 2560 scope.go:117] "RemoveContainer" containerID="04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d" Jul 24 15:38:50.040723 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.040703 2560 scope.go:117] "RemoveContainer" containerID="2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7" Jul 24 15:38:50.048218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.048199 2560 scope.go:117] "RemoveContainer" containerID="13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926" Jul 24 15:38:50.055881 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.055864 2560 scope.go:117] "RemoveContainer" containerID="326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53" Jul 24 15:38:50.056144 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:50.056124 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53\": container with ID starting with 326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53 not found: ID does not exist" containerID="326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53" Jul 24 15:38:50.056194 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.056154 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53"} err="failed to get container status \"326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53\": rpc error: code = NotFound desc = could not find container \"326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53\": container with ID starting with 326dcd6fd035b4d31fb01590689406ad13e33b1e58a27af911c631994dd86b53 not found: ID does not exist" Jul 24 15:38:50.056194 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.056188 2560 scope.go:117] "RemoveContainer" containerID="3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962" Jul 24 15:38:50.056619 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:50.056587 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962\": container with ID starting with 3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962 not found: ID does not exist" containerID="3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962" Jul 24 15:38:50.056776 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.056637 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962"} err="failed to get container status \"3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962\": rpc error: code = NotFound desc = could not find container \"3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962\": container with ID starting with 3245ceabc8a575b6032680cb7e47f70144fb3810a3eacc91aa32e51bf1ed0962 not found: ID does not exist" Jul 24 15:38:50.056776 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.056658 2560 scope.go:117] "RemoveContainer" containerID="7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169" Jul 24 15:38:50.057161 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:50.057058 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169\": container with ID starting with 7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169 not found: ID does not exist" containerID="7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169" Jul 24 15:38:50.057241 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.057172 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169"} err="failed to get container status \"7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169\": rpc error: code = NotFound desc = could not find container \"7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169\": container with ID starting with 7f9aec4da1cd5513b3e59e0309dc650d6b4f6dedb8232ab40c8397539ea94169 not found: ID does not exist" Jul 24 15:38:50.057241 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.057193 2560 scope.go:117] "RemoveContainer" containerID="ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1" Jul 24 15:38:50.057525 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:50.057498 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1\": container with ID starting with ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1 not found: ID does not exist" containerID="ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1" Jul 24 15:38:50.057617 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.057533 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1"} err="failed to get container status \"ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1\": rpc error: code = NotFound desc = could not find container \"ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1\": container with ID starting with ba4be52a2c0dd23dd35d12fc9db0da59d1e7f1fc941514931ac5317993d5d4a1 not found: ID does not exist" Jul 24 15:38:50.057617 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.057553 2560 scope.go:117] "RemoveContainer" containerID="04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d" Jul 24 15:38:50.057834 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:50.057816 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d\": container with ID starting with 04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d not found: ID does not exist" containerID="04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d" Jul 24 15:38:50.057874 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.057838 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d"} err="failed to get container status \"04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d\": rpc error: code = NotFound desc = could not find container \"04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d\": container with ID starting with 04da43375777a4ea33fdf0f08abd7b6658db60e1aec202ebfa7d2a721e2b584d not found: ID does not exist" Jul 24 15:38:50.057874 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.057853 2560 scope.go:117] "RemoveContainer" containerID="2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7" Jul 24 15:38:50.058106 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:50.058089 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7\": container with ID starting with 2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7 not found: ID does not exist" containerID="2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7" Jul 24 15:38:50.058180 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.058112 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7"} err="failed to get container status \"2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7\": rpc error: code = NotFound desc = could not find container \"2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7\": container with ID starting with 2821cb2c6924c6b1b7281c0b7f67b03add5a0a4a303e7e540bb1e7b8802677f7 not found: ID does not exist" Jul 24 15:38:50.058180 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.058132 2560 scope.go:117] "RemoveContainer" containerID="13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926" Jul 24 15:38:50.058416 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:50.058399 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926\": container with ID starting with 13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926 not found: ID does not exist" containerID="13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926" Jul 24 15:38:50.058465 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.058427 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926"} err="failed to get container status \"13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926\": rpc error: code = NotFound desc = could not find container \"13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926\": container with ID starting with 13e9a406cb394dae4e4489364125bdf0b14a064692eb6cf6cac3fabda87ef926 not found: ID does not exist" Jul 24 15:38:50.069698 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.069672 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:38:50.070009 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.069995 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="config-reloader" Jul 24 15:38:50.070061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070012 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="config-reloader" Jul 24 15:38:50.070061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070031 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-metric" Jul 24 15:38:50.070061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070038 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-metric" Jul 24 15:38:50.070061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070045 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="init-config-reloader" Jul 24 15:38:50.070061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070051 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="init-config-reloader" Jul 24 15:38:50.070061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070057 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy" Jul 24 15:38:50.070061 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070062 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070069 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-web" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070074 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-web" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070082 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="alertmanager" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070088 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="alertmanager" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070093 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="prom-label-proxy" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070098 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="prom-label-proxy" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070144 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070151 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-metric" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070157 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="config-reloader" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070164 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="alertmanager" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070170 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="prom-label-proxy" Jul 24 15:38:50.070265 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.070176 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" containerName="kube-rbac-proxy-web" Jul 24 15:38:50.073793 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.073775 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.076416 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076318 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 15:38:50.076525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076414 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qn5tk\"" Jul 24 15:38:50.076525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076447 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 15:38:50.076525 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076474 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 15:38:50.076747 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076727 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 15:38:50.076801 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076742 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 15:38:50.076852 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076729 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 15:38:50.076852 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076861 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 15:38:50.077009 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.076966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 15:38:50.082426 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.082406 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 15:38:50.091888 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.091863 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:38:50.164225 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164225 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164229 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-web-config\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164467 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164467 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164467 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164331 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-config-out\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164467 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164360 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164467 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164467 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164403 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164467 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs2f5\" (UniqueName: \"kubernetes.io/projected/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-kube-api-access-hs2f5\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164471 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164490 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164561 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.164677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.164596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.265599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.265562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs2f5\" (UniqueName: \"kubernetes.io/projected/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-kube-api-access-hs2f5\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.265729 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.265610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.265729 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.265654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.265808 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.265728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.265808 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.265773 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266055 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266169 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-web-config\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266169 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266184 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-config-out\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266495 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266495 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266607 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266548 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266983 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.266983 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.266747 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.269226 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.269202 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-web-config\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.269463 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.269441 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-config-out\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.269550 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.269488 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.269829 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.269773 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.269963 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.269943 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.270129 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.270110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.270168 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.270108 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.270321 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.270279 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.271450 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.271433 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.276122 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.276098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs2f5\" (UniqueName: \"kubernetes.io/projected/a4182968-f3dd-4f22-b9a1-a35b7e3ab797-kube-api-access-hs2f5\") pod \"alertmanager-main-0\" (UID: \"a4182968-f3dd-4f22-b9a1-a35b7e3ab797\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.384053 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.384015 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 15:38:50.536954 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.536876 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 15:38:50.540384 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:38:50.540358 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4182968_f3dd_4f22_b9a1_a35b7e3ab797.slice/crio-5c71722bdf7db24e3f2098a1c11f1fb31eb992743165a5f0eb4b2bc895595c6e WatchSource:0}: Error finding container 5c71722bdf7db24e3f2098a1c11f1fb31eb992743165a5f0eb4b2bc895595c6e: Status 404 returned error can't find the container with id 5c71722bdf7db24e3f2098a1c11f1fb31eb992743165a5f0eb4b2bc895595c6e Jul 24 15:38:50.636606 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.636576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:50.739212 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:50.739147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:51.007203 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:51.007115 2560 generic.go:356] "Generic (PLEG): container finished" podID="a4182968-f3dd-4f22-b9a1-a35b7e3ab797" containerID="568f115d5470bfc1439bb03c403cf93dea107dd7f6698ade62e27ca059acec2d" exitCode=0 Jul 24 15:38:51.007610 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:51.007206 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerDied","Data":"568f115d5470bfc1439bb03c403cf93dea107dd7f6698ade62e27ca059acec2d"} Jul 24 15:38:51.007610 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:51.007323 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerStarted","Data":"5c71722bdf7db24e3f2098a1c11f1fb31eb992743165a5f0eb4b2bc895595c6e"} Jul 24 15:38:51.140464 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:51.140433 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ae215b3-f896-4504-a14b-6fc2b187fc81" path="/var/lib/kubelet/pods/5ae215b3-f896-4504-a14b-6fc2b187fc81/volumes" Jul 24 15:38:52.015828 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.015792 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerStarted","Data":"75566dd90528a925c2075257a94baf10e1a848b507d95794b66f25565e88723a"} Jul 24 15:38:52.016206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.015864 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerStarted","Data":"bcf4a84fbd7627087cf9ea999a2583e0498421349310ed34e684b0561dd42696"} Jul 24 15:38:52.016206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.015889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerStarted","Data":"9ce1edb4e0861c633612b97612f09969b5743d39b3665a439c9387016cc8af9c"} Jul 24 15:38:52.016206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.015928 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerStarted","Data":"0c52ef22e3d9bc29537a49d4867e7c4463f25d8505fb4afe6f8c24eefd950837"} Jul 24 15:38:52.016206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.015956 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerStarted","Data":"78ad9cf7cdfc5d086140a4d0cc537fcc285d66545a63635d3e949a2575ffe0ad"} Jul 24 15:38:52.016206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.015979 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4182968-f3dd-4f22-b9a1-a35b7e3ab797","Type":"ContainerStarted","Data":"6a395a6288160759077991d7d5f338cf2eace4d44c2fe9174d5fe6c393a70c23"} Jul 24 15:38:52.045808 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.045739 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.030975847 podStartE2EDuration="2.045717642s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.741795ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:38:52.044551504 +0000 UTC m=+155.467152553" watchObservedRunningTime="2026-07-24 15:38:52.045717642 +0000 UTC m=+155.468318671" Jul 24 15:38:52.108370 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.108332 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:38:52.108742 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.108707 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="prometheus" containerID="cri-o://782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8" gracePeriod=600 Jul 24 15:38:52.108853 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.108732 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy" containerID="cri-o://dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0" gracePeriod=600 Jul 24 15:38:52.108853 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.108770 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-web" containerID="cri-o://281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3" gracePeriod=600 Jul 24 15:38:52.108853 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.108796 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="config-reloader" containerID="cri-o://e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c" gracePeriod=600 Jul 24 15:38:52.108853 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.108772 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="thanos-sidecar" containerID="cri-o://a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f" gracePeriod=600 Jul 24 15:38:52.109075 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:52.109052 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce" gracePeriod=600 Jul 24 15:38:53.027992 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.027958 2560 generic.go:356] "Generic (PLEG): container finished" podID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerID="f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce" exitCode=0 Jul 24 15:38:53.027992 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.027985 2560 generic.go:356] "Generic (PLEG): container finished" podID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerID="dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0" exitCode=0 Jul 24 15:38:53.027992 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.027991 2560 generic.go:356] "Generic (PLEG): container finished" podID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerID="a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f" exitCode=0 Jul 24 15:38:53.027992 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.027997 2560 generic.go:356] "Generic (PLEG): container finished" podID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerID="e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c" exitCode=0 Jul 24 15:38:53.028478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.028009 2560 generic.go:356] "Generic (PLEG): container finished" podID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerID="782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8" exitCode=0 Jul 24 15:38:53.028478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.028028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce"} Jul 24 15:38:53.028478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.028131 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0"} Jul 24 15:38:53.028478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.028171 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f"} Jul 24 15:38:53.028478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.028207 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c"} Jul 24 15:38:53.028478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.028242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8"} Jul 24 15:38:53.365544 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.365518 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:53.498206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498169 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-kube-rbac-proxy\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498212 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-metrics-client-certs\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498243 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-kubelet-serving-ca-bundle\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498286 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-config\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498336 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-0\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498398 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-metrics-client-ca\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498426 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-web-config\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498457 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-db\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498775 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498492 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498775 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498650 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498775 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498699 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-tls\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498775 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498725 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-1\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.498775 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498757 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498778 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-thanos-prometheus-http-client-file\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498815 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp2dm\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-kube-api-access-wp2dm\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498850 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-grpc-tls\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498880 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-serving-certs-ca-bundle\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498903 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-trusted-ca-bundle\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498934 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-tls-assets\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498968 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-2\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499012 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.498992 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-config-out\") pod \"e049b69d-59a7-4279-ae6e-f324b59e09bb\" (UID: \"e049b69d-59a7-4279-ae6e-f324b59e09bb\") " Jul 24 15:38:53.499514 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.499491 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.500163 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.500132 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:53.500331 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.500152 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:38:53.501173 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.501140 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:53.501329 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.501198 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-config" (OuterVolumeSpecName: "config") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.501329 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.501229 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.501640 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.501610 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:53.501640 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.501628 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.502377 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.501661 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:53.502377 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.501866 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-config-out" (OuterVolumeSpecName: "config-out") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:38:53.502377 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.502044 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:53.502377 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.502324 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.503555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.502548 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:38:53.503555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.503271 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.503877 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.503846 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.504103 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.504078 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.504271 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.504246 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.504271 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.504263 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:38:53.504546 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.504530 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-kube-api-access-wp2dm" (OuterVolumeSpecName: "kube-api-access-wp2dm") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "kube-api-access-wp2dm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:38:53.514540 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.514511 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-web-config" (OuterVolumeSpecName: "web-config") pod "e049b69d-59a7-4279-ae6e-f324b59e09bb" (UID: "e049b69d-59a7-4279-ae6e-f324b59e09bb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600534 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-metrics-client-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600565 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-config\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600575 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600584 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-metrics-client-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600593 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-web-config\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600601 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-db\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600610 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600619 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600621 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600630 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600638 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600646 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600655 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp2dm\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-kube-api-access-wp2dm\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600662 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-grpc-tls\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600670 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600678 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600685 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e049b69d-59a7-4279-ae6e-f324b59e09bb-tls-assets\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600692 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e049b69d-59a7-4279-ae6e-f324b59e09bb-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600699 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e049b69d-59a7-4279-ae6e-f324b59e09bb-config-out\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:53.600955 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:53.600707 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e049b69d-59a7-4279-ae6e-f324b59e09bb-secret-kube-rbac-proxy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 15:38:54.035237 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.035203 2560 generic.go:356] "Generic (PLEG): container finished" podID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerID="281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3" exitCode=0 Jul 24 15:38:54.035651 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.035275 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3"} Jul 24 15:38:54.035651 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.035366 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e049b69d-59a7-4279-ae6e-f324b59e09bb","Type":"ContainerDied","Data":"2ae3a4561a9279fc26476486e92026964e29b3efcf70847ad1e86406bcc7c5cd"} Jul 24 15:38:54.035651 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.035280 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.035651 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.035405 2560 scope.go:117] "RemoveContainer" containerID="f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce" Jul 24 15:38:54.044370 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.044343 2560 scope.go:117] "RemoveContainer" containerID="dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0" Jul 24 15:38:54.052116 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.052093 2560 scope.go:117] "RemoveContainer" containerID="281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3" Jul 24 15:38:54.059920 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.059880 2560 scope.go:117] "RemoveContainer" containerID="a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f" Jul 24 15:38:54.063054 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.063033 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:38:54.066870 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.066845 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:38:54.074064 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.074044 2560 scope.go:117] "RemoveContainer" containerID="e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c" Jul 24 15:38:54.081842 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.081820 2560 scope.go:117] "RemoveContainer" containerID="782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8" Jul 24 15:38:54.089872 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.089850 2560 scope.go:117] "RemoveContainer" containerID="eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8" Jul 24 15:38:54.097092 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.097071 2560 scope.go:117] "RemoveContainer" containerID="f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce" Jul 24 15:38:54.097392 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:54.097362 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce\": container with ID starting with f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce not found: ID does not exist" containerID="f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce" Jul 24 15:38:54.097478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.097398 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce"} err="failed to get container status \"f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce\": rpc error: code = NotFound desc = could not find container \"f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce\": container with ID starting with f36da33f13ab427d0a63ca73909fd5820ecd4f5a97a99e06c23327aa23a688ce not found: ID does not exist" Jul 24 15:38:54.097478 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.097427 2560 scope.go:117] "RemoveContainer" containerID="dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0" Jul 24 15:38:54.097733 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:54.097714 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0\": container with ID starting with dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0 not found: ID does not exist" containerID="dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0" Jul 24 15:38:54.097774 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.097740 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0"} err="failed to get container status \"dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0\": rpc error: code = NotFound desc = could not find container \"dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0\": container with ID starting with dda17cb581fd95383e0b39cea6350821793d683fbb8fbe93339f52266a5d93b0 not found: ID does not exist" Jul 24 15:38:54.097774 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.097758 2560 scope.go:117] "RemoveContainer" containerID="281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3" Jul 24 15:38:54.097990 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:54.097973 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3\": container with ID starting with 281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3 not found: ID does not exist" containerID="281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3" Jul 24 15:38:54.098032 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.097995 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3"} err="failed to get container status \"281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3\": rpc error: code = NotFound desc = could not find container \"281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3\": container with ID starting with 281daee935bef3522a5bb43f09e6aff7316e9e21f25096ae9324d000a9eec2a3 not found: ID does not exist" Jul 24 15:38:54.098032 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.098011 2560 scope.go:117] "RemoveContainer" containerID="a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f" Jul 24 15:38:54.098262 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:54.098245 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f\": container with ID starting with a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f not found: ID does not exist" containerID="a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f" Jul 24 15:38:54.098341 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.098269 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f"} err="failed to get container status \"a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f\": rpc error: code = NotFound desc = could not find container \"a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f\": container with ID starting with a89b57f4b5e430c1bf18cae22b30544e8b4064f56f3eaeee566b125d1ba83a2f not found: ID does not exist" Jul 24 15:38:54.098341 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.098288 2560 scope.go:117] "RemoveContainer" containerID="e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c" Jul 24 15:38:54.098552 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:54.098532 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c\": container with ID starting with e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c not found: ID does not exist" containerID="e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c" Jul 24 15:38:54.098602 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.098556 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c"} err="failed to get container status \"e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c\": rpc error: code = NotFound desc = could not find container \"e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c\": container with ID starting with e9263e7b7ea4e12d42d48d61f29e0cbefecc274cbfd81786e262bca07eac218c not found: ID does not exist" Jul 24 15:38:54.098602 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.098572 2560 scope.go:117] "RemoveContainer" containerID="782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8" Jul 24 15:38:54.098777 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:54.098759 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8\": container with ID starting with 782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8 not found: ID does not exist" containerID="782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8" Jul 24 15:38:54.098841 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.098785 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8"} err="failed to get container status \"782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8\": rpc error: code = NotFound desc = could not find container \"782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8\": container with ID starting with 782f7ba0557023de8b4956f4efa297bbdf8e60f007822fc7469aa8bf41cb78a8 not found: ID does not exist" Jul 24 15:38:54.098841 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.098804 2560 scope.go:117] "RemoveContainer" containerID="eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8" Jul 24 15:38:54.099036 ip-10-0-132-149 kubenswrapper[2560]: E0724 15:38:54.099021 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8\": container with ID starting with eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8 not found: ID does not exist" containerID="eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8" Jul 24 15:38:54.099078 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.099039 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8"} err="failed to get container status \"eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8\": rpc error: code = NotFound desc = could not find container \"eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8\": container with ID starting with eb2eefd3832bf16ef1f0daa225231407083781dfcb2ba0f4d46700053af461b8 not found: ID does not exist" Jul 24 15:38:54.101610 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101591 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:38:54.101917 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101905 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-web" Jul 24 15:38:54.101965 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101920 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-web" Jul 24 15:38:54.101965 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101937 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy" Jul 24 15:38:54.101965 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101945 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy" Jul 24 15:38:54.101965 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101955 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="config-reloader" Jul 24 15:38:54.101965 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101962 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="config-reloader" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101978 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="prometheus" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101985 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="prometheus" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.101997 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="thanos-sidecar" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102002 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="thanos-sidecar" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102009 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="init-config-reloader" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102014 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="init-config-reloader" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102019 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-thanos" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102024 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-thanos" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102072 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-web" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102081 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102086 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="prometheus" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102091 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="thanos-sidecar" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102097 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="config-reloader" Jul 24 15:38:54.102147 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.102102 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" containerName="kube-rbac-proxy-thanos" Jul 24 15:38:54.105636 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.105620 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.109010 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.108989 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 24 15:38:54.109134 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.109115 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 24 15:38:54.109287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.109136 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 24 15:38:54.110035 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.110012 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 24 15:38:54.110035 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.110027 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 24 15:38:54.110141 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.110030 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mrd66\"" Jul 24 15:38:54.114980 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.114958 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 24 15:38:54.115132 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.115094 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 24 15:38:54.115212 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.115124 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 24 15:38:54.115212 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.115150 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 24 15:38:54.115212 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.115173 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 24 15:38:54.115435 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.115251 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bfmvn529bl84h\"" Jul 24 15:38:54.115435 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.115278 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 24 15:38:54.115569 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.115556 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 24 15:38:54.120972 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.120956 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 24 15:38:54.123911 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.123896 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 24 15:38:54.134959 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.134930 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:38:54.206239 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206198 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-web-config\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206239 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206546 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206546 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206403 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206546 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r79hm\" (UniqueName: \"kubernetes.io/projected/ed843ba5-2d7a-4904-afc5-860bdcb57859-kube-api-access-r79hm\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206546 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206546 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206532 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed843ba5-2d7a-4904-afc5-860bdcb57859-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206605 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-config\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206656 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206690 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206776 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.206857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed843ba5-2d7a-4904-afc5-860bdcb57859-config-out\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.207142 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.207142 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206908 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.207142 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.206976 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.207142 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.207006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308087 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.307996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed843ba5-2d7a-4904-afc5-860bdcb57859-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308087 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308087 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-config\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308087 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308113 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed843ba5-2d7a-4904-afc5-860bdcb57859-config-out\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308495 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-web-config\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r79hm\" (UniqueName: \"kubernetes.io/projected/ed843ba5-2d7a-4904-afc5-860bdcb57859-kube-api-access-r79hm\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.308833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.308710 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.309466 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.309182 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.310043 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.310016 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.310685 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.310666 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.311440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.311331 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed843ba5-2d7a-4904-afc5-860bdcb57859-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.311440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.311411 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-config\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.312382 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.311755 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.312382 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.311798 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.312382 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.311871 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.312382 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.312015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.312606 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.312473 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.312606 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.312528 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.313469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.313110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed843ba5-2d7a-4904-afc5-860bdcb57859-config-out\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.313469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.313129 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ed843ba5-2d7a-4904-afc5-860bdcb57859-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.313469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.313396 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.313469 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.313407 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.314756 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.314731 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-web-config\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.314838 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.314731 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.314838 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.314821 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.315021 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.314999 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ed843ba5-2d7a-4904-afc5-860bdcb57859-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.323199 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.323179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r79hm\" (UniqueName: \"kubernetes.io/projected/ed843ba5-2d7a-4904-afc5-860bdcb57859-kube-api-access-r79hm\") pod \"prometheus-k8s-0\" (UID: \"ed843ba5-2d7a-4904-afc5-860bdcb57859\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.414788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.414749 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:38:54.568242 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:54.568160 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 24 15:38:54.570896 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:38:54.570867 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded843ba5_2d7a_4904_afc5_860bdcb57859.slice/crio-48d185a3e11d6e20d4ce4c8064d431ca35336a8df18f040e04725f8d7edf4b00 WatchSource:0}: Error finding container 48d185a3e11d6e20d4ce4c8064d431ca35336a8df18f040e04725f8d7edf4b00: Status 404 returned error can't find the container with id 48d185a3e11d6e20d4ce4c8064d431ca35336a8df18f040e04725f8d7edf4b00 Jul 24 15:38:55.041981 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:55.041939 2560 generic.go:356] "Generic (PLEG): container finished" podID="ed843ba5-2d7a-4904-afc5-860bdcb57859" containerID="36a38c311f788fbb75c97de58f683467b050e0c5b7f537dd6fc46102d8696e39" exitCode=0 Jul 24 15:38:55.042413 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:55.041985 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerDied","Data":"36a38c311f788fbb75c97de58f683467b050e0c5b7f537dd6fc46102d8696e39"} Jul 24 15:38:55.042413 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:55.042054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerStarted","Data":"48d185a3e11d6e20d4ce4c8064d431ca35336a8df18f040e04725f8d7edf4b00"} Jul 24 15:38:55.141704 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:55.141668 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e049b69d-59a7-4279-ae6e-f324b59e09bb" path="/var/lib/kubelet/pods/e049b69d-59a7-4279-ae6e-f324b59e09bb/volumes" Jul 24 15:38:55.392812 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:55.392771 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:55.493654 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:55.493597 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:56.050031 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:56.049995 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerStarted","Data":"d2a241ad39614cc9e974486c751ee6afe157b723eb74d8300be713b945007005"} Jul 24 15:38:56.050464 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:56.050063 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerStarted","Data":"662c3e18b11e24e01e61042f90bc6d1f5d9bf6551a50af45f321893e93f187e2"} Jul 24 15:38:56.050464 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:56.050091 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerStarted","Data":"441acc85c5e3f07c5eb2903d527fd122dadfdf480d974b94f0cee3bce93d7d5f"} Jul 24 15:38:56.050464 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:56.050117 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerStarted","Data":"8fb92cb2a04f5b00795eece8342808304a1e2b9ce3210378f762080a7c89f2d7"} Jul 24 15:38:56.050464 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:56.050139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerStarted","Data":"bfee1919e2fa967999721e2f6a5b7430417962ed67d69916311602bbf741d168"} Jul 24 15:38:56.050464 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:56.050162 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ed843ba5-2d7a-4904-afc5-860bdcb57859","Type":"ContainerStarted","Data":"b4202520d8232bd4696cf281f5df3ab3fda2daff6ae18f5594165230661d093b"} Jul 24 15:38:56.081798 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:56.081734 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.065356385 podStartE2EDuration="2.081712796s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.356411ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:38:56.079187399 +0000 UTC m=+159.501788455" watchObservedRunningTime="2026-07-24 15:38:56.081712796 +0000 UTC m=+159.504313829" Jul 24 15:38:59.415836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:38:59.415795 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:39:01.268536 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:01.268502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:01.333491 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:01.333454 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:01.399830 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:01.399797 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:08.020638 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:08.020551 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:11.722221 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:11.722184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:11.843281 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:11.843249 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:20.612881 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:20.612845 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:49.095496 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:49.095458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:52.959821 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:52.959780 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:53.057833 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:53.057799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:54.415185 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:54.415152 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:39:54.431771 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:54.431745 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:39:55.307156 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:39:55.307126 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 24 15:40:19.755798 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:19.755761 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:20.747651 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:20.747608 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:23.396196 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:23.396146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:23.497663 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:23.497626 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:28.548394 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:28.548356 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:28.605055 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:28.605028 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:36.932972 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:36.932938 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:37.054988 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.054921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:37.160163 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.160125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:37.258905 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.258826 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mt5t6"] Jul 24 15:40:37.262084 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.262066 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.265547 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.265527 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 15:40:37.265912 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.265891 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7db9m\"" Jul 24 15:40:37.265912 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.265908 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 15:40:37.283137 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.283100 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mt5t6"] Jul 24 15:40:37.435795 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.435743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mt5t6\" (UID: \"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.435988 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.435809 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbhw5\" (UniqueName: \"kubernetes.io/projected/a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d-kube-api-access-hbhw5\") pod \"cert-manager-webhook-6486ffd997-mt5t6\" (UID: \"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.536259 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.536224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mt5t6\" (UID: \"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.536461 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.536271 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbhw5\" (UniqueName: \"kubernetes.io/projected/a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d-kube-api-access-hbhw5\") pod \"cert-manager-webhook-6486ffd997-mt5t6\" (UID: \"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.548042 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.548009 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mt5t6\" (UID: \"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.548170 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.548062 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbhw5\" (UniqueName: \"kubernetes.io/projected/a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d-kube-api-access-hbhw5\") pod \"cert-manager-webhook-6486ffd997-mt5t6\" (UID: \"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.585040 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.585007 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:37.742998 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:37.742939 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mt5t6"] Jul 24 15:40:37.745788 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:40:37.745755 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6bf6468_bd63_42cc_b2b0_18af5a3e3a5d.slice/crio-39848e37255aa31aabd0698bc84543960014c97f6ec33f0832f446885107446d WatchSource:0}: Error finding container 39848e37255aa31aabd0698bc84543960014c97f6ec33f0832f446885107446d: Status 404 returned error can't find the container with id 39848e37255aa31aabd0698bc84543960014c97f6ec33f0832f446885107446d Jul 24 15:40:38.469719 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:38.469673 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" event={"ID":"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d","Type":"ContainerStarted","Data":"39848e37255aa31aabd0698bc84543960014c97f6ec33f0832f446885107446d"} Jul 24 15:40:39.413071 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:39.413038 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:39.821104 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:39.821059 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:45.500615 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:45.500580 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" event={"ID":"a6bf6468-bd63-42cc-b2b0-18af5a3e3a5d","Type":"ContainerStarted","Data":"4b77db1659321134abfd749151d339ff69ae1182ba174426f13e5ad322abb492"} Jul 24 15:40:45.500996 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:45.500637 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:45.518382 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:45.518333 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" podStartSLOduration=1.588396571 podStartE2EDuration="8.518318192s" totalImagesPullingTime="6.929921621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:40:45.51726295 +0000 UTC m=+268.939863980" watchObservedRunningTime="2026-07-24 15:40:45.518318192 +0000 UTC m=+268.940919214" Jul 24 15:40:47.030767 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:47.030729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:48.275113 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.275078 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xsqc8"] Jul 24 15:40:48.278504 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.278487 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.280969 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.280948 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-s4gv6\"" Jul 24 15:40:48.287483 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.287459 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xsqc8"] Jul 24 15:40:48.324035 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.324004 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94b44a8e-bd0b-49d3-a5c7-3d77970a29a0-bound-sa-token\") pod \"cert-manager-55b4d954d5-xsqc8\" (UID: \"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0\") " pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.324170 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.324077 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzqdg\" (UniqueName: \"kubernetes.io/projected/94b44a8e-bd0b-49d3-a5c7-3d77970a29a0-kube-api-access-dzqdg\") pod \"cert-manager-55b4d954d5-xsqc8\" (UID: \"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0\") " pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.424780 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.424738 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94b44a8e-bd0b-49d3-a5c7-3d77970a29a0-bound-sa-token\") pod \"cert-manager-55b4d954d5-xsqc8\" (UID: \"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0\") " pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.424957 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.424817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzqdg\" (UniqueName: \"kubernetes.io/projected/94b44a8e-bd0b-49d3-a5c7-3d77970a29a0-kube-api-access-dzqdg\") pod \"cert-manager-55b4d954d5-xsqc8\" (UID: \"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0\") " pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.433599 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.433572 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/94b44a8e-bd0b-49d3-a5c7-3d77970a29a0-bound-sa-token\") pod \"cert-manager-55b4d954d5-xsqc8\" (UID: \"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0\") " pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.433711 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.433676 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzqdg\" (UniqueName: \"kubernetes.io/projected/94b44a8e-bd0b-49d3-a5c7-3d77970a29a0-kube-api-access-dzqdg\") pod \"cert-manager-55b4d954d5-xsqc8\" (UID: \"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0\") " pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.588008 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.587925 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xsqc8" Jul 24 15:40:48.722200 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:48.722169 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xsqc8"] Jul 24 15:40:48.724708 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:40:48.724671 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94b44a8e_bd0b_49d3_a5c7_3d77970a29a0.slice/crio-2a641d961537f855a96e628a9661ee646d043ac07692581a9eb30acae342442b WatchSource:0}: Error finding container 2a641d961537f855a96e628a9661ee646d043ac07692581a9eb30acae342442b: Status 404 returned error can't find the container with id 2a641d961537f855a96e628a9661ee646d043ac07692581a9eb30acae342442b Jul 24 15:40:49.255248 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:49.255212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:49.315434 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:49.315400 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:49.517507 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:49.517472 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xsqc8" event={"ID":"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0","Type":"ContainerStarted","Data":"a46625ac8e63648ecf9be560001bf068ef0ba894a75d5434f070520926924e08"} Jul 24 15:40:49.517507 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:49.517510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xsqc8" event={"ID":"94b44a8e-bd0b-49d3-a5c7-3d77970a29a0","Type":"ContainerStarted","Data":"2a641d961537f855a96e628a9661ee646d043ac07692581a9eb30acae342442b"} Jul 24 15:40:49.535977 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:49.535921 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xsqc8" podStartSLOduration=1.5359046159999998 podStartE2EDuration="1.535904616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:40:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:40:49.53518292 +0000 UTC m=+272.957783943" watchObservedRunningTime="2026-07-24 15:40:49.535904616 +0000 UTC m=+272.958505665" Jul 24 15:40:50.238813 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:50.238772 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:50.342653 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:50.342616 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:51.507127 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:51.507095 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mt5t6" Jul 24 15:40:55.961099 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:55.961059 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:56.059664 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:56.059615 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:58.318795 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:40:58.318764 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:09.797190 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:09.797153 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:10.826338 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.826291 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh"] Jul 24 15:41:10.829751 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.829729 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:10.832365 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.832339 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 15:41:10.832510 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.832372 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 15:41:10.832510 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.832443 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 15:41:10.832510 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.832501 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bk4ws\"" Jul 24 15:41:10.832812 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.832794 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 15:41:10.845116 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.845091 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh"] Jul 24 15:41:10.933364 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.933321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/39ac11c7-c954-4499-8b76-fac6954bc85d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:10.933364 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.933367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxdrj\" (UniqueName: \"kubernetes.io/projected/39ac11c7-c954-4499-8b76-fac6954bc85d-kube-api-access-xxdrj\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:10.933625 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.933389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39ac11c7-c954-4499-8b76-fac6954bc85d-tmp\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:10.933625 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.933407 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/39ac11c7-c954-4499-8b76-fac6954bc85d-manifests\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:10.933625 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:10.933529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39ac11c7-c954-4499-8b76-fac6954bc85d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.034378 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.034332 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/39ac11c7-c954-4499-8b76-fac6954bc85d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.034378 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.034380 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxdrj\" (UniqueName: \"kubernetes.io/projected/39ac11c7-c954-4499-8b76-fac6954bc85d-kube-api-access-xxdrj\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.034646 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.034516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39ac11c7-c954-4499-8b76-fac6954bc85d-tmp\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.034646 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.034551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/39ac11c7-c954-4499-8b76-fac6954bc85d-manifests\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.034748 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.034652 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39ac11c7-c954-4499-8b76-fac6954bc85d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.035019 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.034991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/39ac11c7-c954-4499-8b76-fac6954bc85d-manifests\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.036851 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.036827 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39ac11c7-c954-4499-8b76-fac6954bc85d-tmp\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.036980 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.036962 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/39ac11c7-c954-4499-8b76-fac6954bc85d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.037225 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.037207 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39ac11c7-c954-4499-8b76-fac6954bc85d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.041954 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.041935 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxdrj\" (UniqueName: \"kubernetes.io/projected/39ac11c7-c954-4499-8b76-fac6954bc85d-kube-api-access-xxdrj\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-n54rh\" (UID: \"39ac11c7-c954-4499-8b76-fac6954bc85d\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.118937 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.118843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:11.139731 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.139695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:11.207790 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.207759 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:11.282112 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.282084 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh"] Jul 24 15:41:11.284803 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:41:11.284766 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ac11c7_c954_4499_8b76_fac6954bc85d.slice/crio-ba81efa39e341a199985e585a723a385fe920ab55f9dd7c5525fbe890acaa442 WatchSource:0}: Error finding container ba81efa39e341a199985e585a723a385fe920ab55f9dd7c5525fbe890acaa442: Status 404 returned error can't find the container with id ba81efa39e341a199985e585a723a385fe920ab55f9dd7c5525fbe890acaa442 Jul 24 15:41:11.284931 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.284837 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:11.355165 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.355131 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:11.456606 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.456521 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:11.559920 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.559887 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:11.606381 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:11.606349 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" event={"ID":"39ac11c7-c954-4499-8b76-fac6954bc85d","Type":"ContainerStarted","Data":"ba81efa39e341a199985e585a723a385fe920ab55f9dd7c5525fbe890acaa442"} Jul 24 15:41:14.623859 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:14.623767 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" event={"ID":"39ac11c7-c954-4499-8b76-fac6954bc85d","Type":"ContainerStarted","Data":"0d24010a894e7d246c5ca9b1b7b0afe02618d4cf5e210a90eb2b1eea6499877b"} Jul 24 15:41:15.629984 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:15.629945 2560 generic.go:356] "Generic (PLEG): container finished" podID="39ac11c7-c954-4499-8b76-fac6954bc85d" containerID="0d24010a894e7d246c5ca9b1b7b0afe02618d4cf5e210a90eb2b1eea6499877b" exitCode=0 Jul 24 15:41:15.630374 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:15.629993 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" event={"ID":"39ac11c7-c954-4499-8b76-fac6954bc85d","Type":"ContainerDied","Data":"0d24010a894e7d246c5ca9b1b7b0afe02618d4cf5e210a90eb2b1eea6499877b"} Jul 24 15:41:16.636615 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:16.636580 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" event={"ID":"39ac11c7-c954-4499-8b76-fac6954bc85d","Type":"ContainerStarted","Data":"8fa571d53a40f4f205060185dbe44d2ff0c33a3b95f7d93812a83aad4081fdde"} Jul 24 15:41:16.637053 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:16.636699 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:16.659604 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:16.659546 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" podStartSLOduration=2.234871311 podStartE2EDuration="6.659528724s" totalImagesPullingTime="3.371532923s" totalInitContainerRuntime="1.05312449s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:16.657284686 +0000 UTC m=+300.079885737" watchObservedRunningTime="2026-07-24 15:41:16.659528724 +0000 UTC m=+300.082129755" Jul 24 15:41:16.973258 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:16.973181 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 15:41:24.503674 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:24.503631 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:24.603933 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:24.603896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:27.643421 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:27.643391 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-n54rh" Jul 24 15:41:29.867567 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:29.867522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:30.018929 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:30.018890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:30.733470 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:30.733430 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:30.937609 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:30.937579 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:31.048440 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.048397 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:31.057052 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.057019 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2"] Jul 24 15:41:31.061482 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.061459 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.066104 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.065846 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 15:41:31.066104 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.065871 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-txgss\"" Jul 24 15:41:31.066104 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.065888 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 15:41:31.066104 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.065917 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 15:41:31.066104 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.065854 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 15:41:31.066104 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.065995 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 15:41:31.072108 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.072084 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2"] Jul 24 15:41:31.109836 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.109806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b021708-a630-4b5a-bfa9-c25fcc81a93d-manager-config\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.109992 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.109850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7psch\" (UniqueName: \"kubernetes.io/projected/5b021708-a630-4b5a-bfa9-c25fcc81a93d-kube-api-access-7psch\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.109992 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.109875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b021708-a630-4b5a-bfa9-c25fcc81a93d-cert\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.109992 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.109945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b021708-a630-4b5a-bfa9-c25fcc81a93d-metrics-cert\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.142751 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.142722 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:31.211001 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.210931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b021708-a630-4b5a-bfa9-c25fcc81a93d-cert\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.211206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.211065 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b021708-a630-4b5a-bfa9-c25fcc81a93d-metrics-cert\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.211283 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.211212 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b021708-a630-4b5a-bfa9-c25fcc81a93d-manager-config\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.211365 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.211327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7psch\" (UniqueName: \"kubernetes.io/projected/5b021708-a630-4b5a-bfa9-c25fcc81a93d-kube-api-access-7psch\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.212015 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.211991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b021708-a630-4b5a-bfa9-c25fcc81a93d-manager-config\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.214275 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.214251 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b021708-a630-4b5a-bfa9-c25fcc81a93d-metrics-cert\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.214491 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.214470 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b021708-a630-4b5a-bfa9-c25fcc81a93d-cert\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.221790 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.221766 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7psch\" (UniqueName: \"kubernetes.io/projected/5b021708-a630-4b5a-bfa9-c25fcc81a93d-kube-api-access-7psch\") pod \"lws-controller-manager-7db5d47bff-pngv2\" (UID: \"5b021708-a630-4b5a-bfa9-c25fcc81a93d\") " pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.372566 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.372458 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:31.516531 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.516486 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2"] Jul 24 15:41:31.525369 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:41:31.525333 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b021708_a630_4b5a_bfa9_c25fcc81a93d.slice/crio-ce51fdd6b34742418a39a2a01695199c1a2a95f7ea146ac28e4f2677a98ae653 WatchSource:0}: Error finding container ce51fdd6b34742418a39a2a01695199c1a2a95f7ea146ac28e4f2677a98ae653: Status 404 returned error can't find the container with id ce51fdd6b34742418a39a2a01695199c1a2a95f7ea146ac28e4f2677a98ae653 Jul 24 15:41:31.527848 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.527823 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:41:31.592062 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.592023 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:31.688518 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.688431 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:31.706972 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.706934 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" event={"ID":"5b021708-a630-4b5a-bfa9-c25fcc81a93d","Type":"ContainerStarted","Data":"ce51fdd6b34742418a39a2a01695199c1a2a95f7ea146ac28e4f2677a98ae653"} Jul 24 15:41:31.930899 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:31.930864 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:32.517912 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:32.517855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:32.608847 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:32.608765 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:33.651442 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:33.651411 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:34.725077 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:34.724976 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" event={"ID":"5b021708-a630-4b5a-bfa9-c25fcc81a93d","Type":"ContainerStarted","Data":"e9d88f04e7a75fe279d5ab4f42e770a229e02ab0c08ec432880f3e801b0457d6"} Jul 24 15:41:34.725077 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:34.725038 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:34.744195 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:34.744135 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" podStartSLOduration=0.908661001 podStartE2EDuration="3.744112872s" totalImagesPullingTime="2.835451871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:34.74087446 +0000 UTC m=+318.163475513" watchObservedRunningTime="2026-07-24 15:41:34.744112872 +0000 UTC m=+318.166713902" Jul 24 15:41:36.738847 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:36.738813 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:36.837266 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:36.837234 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:37.389219 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.389185 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp"] Jul 24 15:41:37.392476 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.392447 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.395177 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.395151 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 15:41:37.395342 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.395158 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 15:41:37.395511 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.395495 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pvmvg\"" Jul 24 15:41:37.395569 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.395510 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 15:41:37.418085 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.418054 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp"] Jul 24 15:41:37.433902 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.433870 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:37.464960 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.464928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465115 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.464977 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lwwc\" (UniqueName: \"kubernetes.io/projected/df4dfeac-3176-418d-a129-3438754d8a18-kube-api-access-7lwwc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465115 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.465005 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465115 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.465029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465115 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.465048 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4dfeac-3176-418d-a129-3438754d8a18-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465115 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.465070 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465414 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.465159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4dfeac-3176-418d-a129-3438754d8a18-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465414 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.465223 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.465414 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.465255 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4dfeac-3176-418d-a129-3438754d8a18-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566205 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566161 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566433 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lwwc\" (UniqueName: \"kubernetes.io/projected/df4dfeac-3176-418d-a129-3438754d8a18-kube-api-access-7lwwc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566433 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566291 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566433 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566433 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566399 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4dfeac-3176-418d-a129-3438754d8a18-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566650 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566465 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566650 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566516 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4dfeac-3176-418d-a129-3438754d8a18-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566650 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566650 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4dfeac-3176-418d-a129-3438754d8a18-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.566863 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.566686 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.567105 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.567079 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.567240 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.567079 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.567367 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.567341 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.567431 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.567363 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df4dfeac-3176-418d-a129-3438754d8a18-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.569385 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.569352 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df4dfeac-3176-418d-a129-3438754d8a18-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.569693 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.569676 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df4dfeac-3176-418d-a129-3438754d8a18-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.582618 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.582586 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lwwc\" (UniqueName: \"kubernetes.io/projected/df4dfeac-3176-418d-a129-3438754d8a18-kube-api-access-7lwwc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.583076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.583052 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df4dfeac-3176-418d-a129-3438754d8a18-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp\" (UID: \"df4dfeac-3176-418d-a129-3438754d8a18\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.588783 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.588751 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:37.704909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.704813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:37.826239 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.826206 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:37.850692 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:37.850557 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp"] Jul 24 15:41:37.853183 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:41:37.853160 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf4dfeac_3176_418d_a129_3438754d8a18.slice/crio-87780c5fe7443acafc7b03bc76614da0a3082ac471eea9e9198e402ecf91946a WatchSource:0}: Error finding container 87780c5fe7443acafc7b03bc76614da0a3082ac471eea9e9198e402ecf91946a: Status 404 returned error can't find the container with id 87780c5fe7443acafc7b03bc76614da0a3082ac471eea9e9198e402ecf91946a Jul 24 15:41:38.744809 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:38.744775 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" event={"ID":"df4dfeac-3176-418d-a129-3438754d8a18","Type":"ContainerStarted","Data":"87780c5fe7443acafc7b03bc76614da0a3082ac471eea9e9198e402ecf91946a"} Jul 24 15:41:39.362726 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.362684 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk"] Jul 24 15:41:39.366496 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.366469 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.369130 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.369105 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 15:41:39.369281 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.369234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wqjj2\"" Jul 24 15:41:39.369674 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.369596 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 15:41:39.369766 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.369752 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 15:41:39.381266 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.381239 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk"] Jul 24 15:41:39.383546 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.383516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926abaa7-73a8-4e1e-827e-184ef782408b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.383677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.383560 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.383677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.383611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.383787 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.383704 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926abaa7-73a8-4e1e-827e-184ef782408b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.383787 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.383768 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.383891 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.383814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrkmk\" (UniqueName: \"kubernetes.io/projected/926abaa7-73a8-4e1e-827e-184ef782408b-kube-api-access-jrkmk\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.383944 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.383914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926abaa7-73a8-4e1e-827e-184ef782408b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.486045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.485341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926abaa7-73a8-4e1e-827e-184ef782408b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.486045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.485424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926abaa7-73a8-4e1e-827e-184ef782408b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.486045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.485455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.486045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.485508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.486045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.485564 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926abaa7-73a8-4e1e-827e-184ef782408b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.486045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.485590 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.486045 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.485631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrkmk\" (UniqueName: \"kubernetes.io/projected/926abaa7-73a8-4e1e-827e-184ef782408b-kube-api-access-jrkmk\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.487036 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.486723 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926abaa7-73a8-4e1e-827e-184ef782408b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.489089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.489036 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926abaa7-73a8-4e1e-827e-184ef782408b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.490106 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.490063 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.490226 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.490198 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.490341 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.490285 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926abaa7-73a8-4e1e-827e-184ef782408b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.499608 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.499578 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926abaa7-73a8-4e1e-827e-184ef782408b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.501160 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.501098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrkmk\" (UniqueName: \"kubernetes.io/projected/926abaa7-73a8-4e1e-827e-184ef782408b-kube-api-access-jrkmk\") pod \"istiod-openshift-gateway-5677ccb49b-ddnwk\" (UID: \"926abaa7-73a8-4e1e-827e-184ef782408b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.678560 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.678474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:39.883096 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:39.883050 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk"] Jul 24 15:41:39.885578 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:41:39.885548 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926abaa7_73a8_4e1e_827e_184ef782408b.slice/crio-aa135bcbd31a6f6a59aef9882d58e1eeffb8d364beaf2e66ac38f49fdaa18c64 WatchSource:0}: Error finding container aa135bcbd31a6f6a59aef9882d58e1eeffb8d364beaf2e66ac38f49fdaa18c64: Status 404 returned error can't find the container with id aa135bcbd31a6f6a59aef9882d58e1eeffb8d364beaf2e66ac38f49fdaa18c64 Jul 24 15:41:40.760220 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:40.760179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" event={"ID":"926abaa7-73a8-4e1e-827e-184ef782408b","Type":"ContainerStarted","Data":"aa135bcbd31a6f6a59aef9882d58e1eeffb8d364beaf2e66ac38f49fdaa18c64"} Jul 24 15:41:41.117940 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.117705 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:41:41.117940 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.117807 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:41:41.117940 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.117849 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:41:41.183187 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.183143 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:41.287020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.286948 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:41.503900 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.503770 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:41.617713 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.617680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:41.719023 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.718976 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:41.768111 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.768072 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" event={"ID":"df4dfeac-3176-418d-a129-3438754d8a18","Type":"ContainerStarted","Data":"c7336b0758ee22b7aef7dd3ea9d4427dd401e3de8c0282680b014919d42814a6"} Jul 24 15:41:41.790565 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.790506 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" podStartSLOduration=1.528479876 podStartE2EDuration="4.79048838s" totalImagesPullingTime="3.262008504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:41.789796836 +0000 UTC m=+325.212397889" watchObservedRunningTime="2026-07-24 15:41:41.79048838 +0000 UTC m=+325.213089408" Jul 24 15:41:41.794626 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:41.794584 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:42.144062 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:42.143978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:42.705578 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:42.705540 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:42.714323 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:42.710932 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:42.773858 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:42.773820 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:42.775135 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:42.775111 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp" Jul 24 15:41:43.003287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:43.003250 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:41:43.003413 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:43.003337 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:41:43.779843 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:43.779803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" event={"ID":"926abaa7-73a8-4e1e-827e-184ef782408b","Type":"ContainerStarted","Data":"ffc898db3df03bce25f57c5bb0fa90192f7e20c4048e950f576a567a3535a24b"} Jul 24 15:41:43.780330 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:43.779883 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:43.828287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:43.828232 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" podStartSLOduration=1.713311375 podStartE2EDuration="4.828212264s" totalImagesPullingTime="3.114900889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:43.826353043 +0000 UTC m=+327.248954093" watchObservedRunningTime="2026-07-24 15:41:43.828212264 +0000 UTC m=+327.250813303" Jul 24 15:41:44.788870 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:44.788832 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ddnwk" Jul 24 15:41:45.732357 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:45.732328 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7db5d47bff-pngv2" Jul 24 15:41:55.057222 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:55.057189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:55.111117 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:41:55.111082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:31.491426 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:31.491385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:31.585968 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:31.585930 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:31.654058 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:31.654025 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:32.410585 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:32.410549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:32.680020 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:32.679938 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:41.080355 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:41.080295 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:41.136064 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:41.136034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:41.221358 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:41.221325 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:41.284829 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:41.284767 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:41.824170 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:41.824112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:41.874588 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:41.874538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:46.080345 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:46.080291 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:46.140771 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:46.140744 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:52.358769 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:52.358730 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:52.444768 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:52.444738 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:52.508009 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:42:52.507971 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:00.864125 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:00.864091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:04.736393 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:04.736354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:06.477699 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:06.477668 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:08.064582 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:08.064542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:08.473079 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:08.472991 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:09.923422 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:09.923385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:11.511108 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:11.511071 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:12.315723 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:12.315686 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:12.411743 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:12.411700 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:14.309051 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:14.309010 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:14.790464 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:14.790417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:14.956957 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:14.956923 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:16.071468 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:16.071438 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:16.185267 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:16.185228 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:24.485352 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:24.485282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:24.596831 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:24.596800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:29.270064 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:29.270019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:29.325425 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:29.325383 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:29.947934 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:29.947900 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:30.018408 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:30.018328 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:33.680751 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:33.680716 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:33.778769 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:33.778733 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:34.452519 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:34.452480 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:39.272739 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:39.272656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:39.328184 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:39.328151 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:45.901409 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:45.901375 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:46.005405 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:46.005369 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:50.704255 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:50.704198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:50.805610 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:50.805558 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:53.184770 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.184733 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:53.242760 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.242717 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:53.372369 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.372316 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk"] Jul 24 15:43:53.374828 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.374807 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.377407 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.377386 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zs9hb\"" Jul 24 15:43:53.388733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.388707 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk"] Jul 24 15:43:53.398532 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.398499 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:53.438765 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.438765 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.438765 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438747 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.439010 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.439010 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e139bc-a01f-40c8-869d-c37b09ca7be4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.439010 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-488jt\" (UniqueName: \"kubernetes.io/projected/85e139bc-a01f-40c8-869d-c37b09ca7be4-kube-api-access-488jt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.439010 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.439010 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.438975 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.439199 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.439056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.540677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.540677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540671 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.540941 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540710 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.540941 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.540941 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540750 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e139bc-a01f-40c8-869d-c37b09ca7be4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.540941 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-488jt\" (UniqueName: \"kubernetes.io/projected/85e139bc-a01f-40c8-869d-c37b09ca7be4-kube-api-access-488jt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541141 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540938 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541141 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.540981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541141 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.541060 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541141 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.541109 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541141 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.541128 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541492 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.541470 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541588 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.541568 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.541733 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.541604 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85e139bc-a01f-40c8-869d-c37b09ca7be4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.543507 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.543484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.543793 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.543776 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.548457 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.548427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:53.549907 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.549888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-488jt\" (UniqueName: \"kubernetes.io/projected/85e139bc-a01f-40c8-869d-c37b09ca7be4-kube-api-access-488jt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.550156 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.550139 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85e139bc-a01f-40c8-869d-c37b09ca7be4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mdfxk\" (UID: \"85e139bc-a01f-40c8-869d-c37b09ca7be4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.685857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.685820 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:53.825165 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.825140 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk"] Jul 24 15:43:53.827180 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:43:53.827145 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e139bc_a01f_40c8_869d_c37b09ca7be4.slice/crio-fc4ef74340bc8b401724491be65f922bff4a2dded13ffae3eed8b81f694c21aa WatchSource:0}: Error finding container fc4ef74340bc8b401724491be65f922bff4a2dded13ffae3eed8b81f694c21aa: Status 404 returned error can't find the container with id fc4ef74340bc8b401724491be65f922bff4a2dded13ffae3eed8b81f694c21aa Jul 24 15:43:53.829141 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.829110 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:43:53.829206 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.829175 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:43:53.829242 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:53.829204 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:43:54.394262 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:54.394228 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" event={"ID":"85e139bc-a01f-40c8-869d-c37b09ca7be4","Type":"ContainerStarted","Data":"fc280ce3b2861d66379ec10001e0b700b5e86d9d31ffdf1457add32d267fbb1c"} Jul 24 15:43:54.394690 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:54.394276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" event={"ID":"85e139bc-a01f-40c8-869d-c37b09ca7be4","Type":"ContainerStarted","Data":"fc4ef74340bc8b401724491be65f922bff4a2dded13ffae3eed8b81f694c21aa"} Jul 24 15:43:54.414222 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:54.414156 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" podStartSLOduration=1.414136429 podStartE2EDuration="1.414136429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:54.413966016 +0000 UTC m=+457.836567089" watchObservedRunningTime="2026-07-24 15:43:54.414136429 +0000 UTC m=+457.836737458" Jul 24 15:43:54.686452 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:54.686358 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:54.690870 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:54.690840 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:55.399550 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.399518 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:55.400590 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.400565 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:43:55.416458 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.416426 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:55.527369 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.527336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:55.582372 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.582335 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:55.813571 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.813536 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:43:55.816105 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.816081 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:55.818348 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.818324 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 24 15:43:55.818502 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.818387 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 15:43:55.818603 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.818586 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 15:43:55.818658 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.818587 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9ds7q\"" Jul 24 15:43:55.827248 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.827225 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:43:55.854463 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.854423 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:43:55.863636 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.863604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98c8b36a-1f80-4e3a-925f-a372b30ff830-config-file\") pod \"limitador-limitador-54d6d988d5-8psn9\" (UID: \"98c8b36a-1f80-4e3a-925f-a372b30ff830\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:55.863636 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.863634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csnjd\" (UniqueName: \"kubernetes.io/projected/98c8b36a-1f80-4e3a-925f-a372b30ff830-kube-api-access-csnjd\") pod \"limitador-limitador-54d6d988d5-8psn9\" (UID: \"98c8b36a-1f80-4e3a-925f-a372b30ff830\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:55.964549 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.964508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98c8b36a-1f80-4e3a-925f-a372b30ff830-config-file\") pod \"limitador-limitador-54d6d988d5-8psn9\" (UID: \"98c8b36a-1f80-4e3a-925f-a372b30ff830\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:55.964549 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.964546 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csnjd\" (UniqueName: \"kubernetes.io/projected/98c8b36a-1f80-4e3a-925f-a372b30ff830-kube-api-access-csnjd\") pod \"limitador-limitador-54d6d988d5-8psn9\" (UID: \"98c8b36a-1f80-4e3a-925f-a372b30ff830\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:55.965178 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.965156 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98c8b36a-1f80-4e3a-925f-a372b30ff830-config-file\") pod \"limitador-limitador-54d6d988d5-8psn9\" (UID: \"98c8b36a-1f80-4e3a-925f-a372b30ff830\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:55.972658 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:55.972624 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csnjd\" (UniqueName: \"kubernetes.io/projected/98c8b36a-1f80-4e3a-925f-a372b30ff830-kube-api-access-csnjd\") pod \"limitador-limitador-54d6d988d5-8psn9\" (UID: \"98c8b36a-1f80-4e3a-925f-a372b30ff830\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:56.127095 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:56.126993 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:56.266610 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:56.266582 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:43:56.268834 ip-10-0-132-149 kubenswrapper[2560]: W0724 15:43:56.268806 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c8b36a_1f80_4e3a_925f_a372b30ff830.slice/crio-fa39a001ad4e2d33570c65194da1734b884b6bc3d293fffd73c7f15d14919855 WatchSource:0}: Error finding container fa39a001ad4e2d33570c65194da1734b884b6bc3d293fffd73c7f15d14919855: Status 404 returned error can't find the container with id fa39a001ad4e2d33570c65194da1734b884b6bc3d293fffd73c7f15d14919855 Jul 24 15:43:56.269131 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:56.269109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:56.333851 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:56.333824 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:56.405677 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:56.405594 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" event={"ID":"98c8b36a-1f80-4e3a-925f-a372b30ff830","Type":"ContainerStarted","Data":"fa39a001ad4e2d33570c65194da1734b884b6bc3d293fffd73c7f15d14919855"} Jul 24 15:43:58.907374 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:58.907345 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:59.008103 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:59.008019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:59.423387 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:59.423347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" event={"ID":"98c8b36a-1f80-4e3a-925f-a372b30ff830","Type":"ContainerStarted","Data":"f20075fd81f6176c5cfe680b084d84fbbff44c8a74e9cc93d4aa69a7f89455a2"} Jul 24 15:43:59.423564 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:59.423508 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:43:59.446413 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:43:59.446366 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" podStartSLOduration=2.010940132 podStartE2EDuration="4.446349789s" totalImagesPullingTime="2.435409657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:59.444431154 +0000 UTC m=+462.867032204" watchObservedRunningTime="2026-07-24 15:43:59.446349789 +0000 UTC m=+462.868950817" Jul 24 15:44:06.904011 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:06.903976 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:07.004778 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:07.004699 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:10.430231 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:10.430194 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8psn9" Jul 24 15:44:11.502258 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:11.502209 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:11.566080 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:11.566046 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:11.631926 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:11.631893 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:11.733936 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:11.733901 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:11.831988 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:11.831958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:12.141203 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:12.141112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:14.910010 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:14.909977 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:15.012909 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:15.012863 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:24.482931 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:24.482898 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:24.708531 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:24.708501 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:31.010531 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:31.010487 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:31.108495 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:31.108462 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:31.177269 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:31.177215 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:31.436400 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:31.436368 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:34.157349 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:34.157291 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:34.253520 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:44:34.253485 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:02.330221 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:02.330184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:09.540902 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:09.540820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:09.638572 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:09.638541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:10.242496 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:10.242453 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:10.635113 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:10.635082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:10.710795 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:10.710761 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:45:10.713076 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:10.713050 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:18.812418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:18.812271 2560 generic.go:356] "Generic (PLEG): container finished" podID="85e139bc-a01f-40c8-869d-c37b09ca7be4" containerID="fc280ce3b2861d66379ec10001e0b700b5e86d9d31ffdf1457add32d267fbb1c" exitCode=0 Jul 24 15:45:18.812418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:18.812344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" event={"ID":"85e139bc-a01f-40c8-869d-c37b09ca7be4","Type":"ContainerDied","Data":"fc280ce3b2861d66379ec10001e0b700b5e86d9d31ffdf1457add32d267fbb1c"} Jul 24 15:45:18.812848 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:18.812731 2560 scope.go:117] "RemoveContainer" containerID="fc280ce3b2861d66379ec10001e0b700b5e86d9d31ffdf1457add32d267fbb1c" Jul 24 15:45:18.813732 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:18.813705 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:45:18.813834 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:18.813768 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:45:18.813834 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:18.813822 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 15:45:19.818969 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:19.818935 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" event={"ID":"85e139bc-a01f-40c8-869d-c37b09ca7be4","Type":"ContainerStarted","Data":"58a3cebf705a29113dc0fd293b10e225e9dea649ce87446d9a51b3a709eef97c"} Jul 24 15:45:20.686900 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:20.686862 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:45:20.688173 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:20.688131 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mdfxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 24 15:45:20.688296 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:20.688197 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" podUID="85e139bc-a01f-40c8-869d-c37b09ca7be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 24 15:45:21.023066 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:21.023033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:21.123568 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:21.123540 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:21.687068 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:21.687037 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mdfxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 24 15:45:21.687240 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:21.687095 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" podUID="85e139bc-a01f-40c8-869d-c37b09ca7be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 24 15:45:22.686233 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:22.686194 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mdfxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 24 15:45:22.686662 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:22.686264 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" podUID="85e139bc-a01f-40c8-869d-c37b09ca7be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 24 15:45:23.690218 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:23.690186 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:45:23.690625 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:23.690525 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:45:23.691294 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:23.691277 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mdfxk" Jul 24 15:45:38.409029 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:38.408997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:38.998019 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:38.997979 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:45:44.219124 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:44.219071 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:45.108101 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:45.108067 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:45:53.997178 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:53.997134 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:55.201605 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:45:55.201563 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:46:11.332640 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:46:11.332604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:12.400781 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:46:12.400737 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:46:18.096752 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:46:18.096718 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:21.107279 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:46:21.107242 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:46:31.630698 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:46:31.630663 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:34.290102 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:46:34.290063 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:47:06.279789 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:06.279752 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:08.286323 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:08.286278 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:26.610551 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:26.610518 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:26.711089 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:26.711042 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:27.784257 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:27.784214 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:27.884553 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:27.884519 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:51.175990 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:51.175948 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:51.276788 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:51.276750 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:58.087587 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:47:58.087552 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:48:08.189848 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:48:08.189756 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:48:12.354418 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:48:12.354371 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:48:12.412998 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:48:12.412969 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:48:18.139801 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:48:18.139762 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:48:54.192950 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:48:54.192886 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:49:04.249824 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:49:04.249781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:49:28.057215 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:49:28.057161 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:49:28.108260 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:49:28.108230 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:49:33.088079 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:49:33.088046 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:49:34.313652 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:49:34.313620 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:01.598857 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:01.598808 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:50:17.190174 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:17.190139 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:50:17.310215 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:17.310163 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:17.412326 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:17.412277 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:17.531545 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:17.531513 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:23.241282 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:23.241240 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:23.312013 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:23.311980 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:23.399188 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:23.399156 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:23.498378 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:23.498269 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:23.599270 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:23.599233 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:29.336892 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:29.336851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:29.436474 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:29.436444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:50:48.191571 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:48.191527 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:50:57.495851 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:50:57.495811 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:51:07.605287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:51:07.605204 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:51:14.995085 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:51:14.995042 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:51:27.491675 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:51:27.491643 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:51:37.494479 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:51:37.494439 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:51:49.896234 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:51:49.896195 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:52:00.396052 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:52:00.396016 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:52:21.691417 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:52:21.691369 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:52:33.792424 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:52:33.792379 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:52:57.095491 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:52:57.095406 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:53:07.390014 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:53:07.389981 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:53:16.324473 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:53:16.324439 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:53:23.297887 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:53:23.297850 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:53:33.397149 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:53:33.397119 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:53:43.704325 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:53:43.704263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:53:53.715640 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:53:53.715603 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:54:03.904886 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:54:03.904841 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:55:20.414490 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:55:20.414430 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:55:39.548969 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:55:39.548847 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:55:49.523021 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:55:49.522986 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:56:00.795395 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:56:00.795348 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:56:23.411748 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:56:23.411710 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:56:36.096532 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:56:36.096499 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:56:46.291818 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:56:46.291779 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:56:58.189286 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:56:58.189246 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:57:06.593702 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:57:06.593660 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:57:16.989022 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:57:16.988976 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:57:24.097665 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:57:24.097623 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:57:28.701198 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:57:28.701162 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:57:36.597287 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:57:36.597250 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:57:53.995042 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:57:53.995003 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:58:03.999772 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:03.999737 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:58:09.829430 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:09.829390 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:58:12.887225 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:12.887189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:58:14.291600 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:14.291559 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:58:27.104517 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:27.104459 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:58:29.596912 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:29.596873 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:58:42.704472 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:42.704388 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:58:43.808555 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:43.808516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 15:58:46.595291 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:58:46.595247 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:59:00.577554 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:59:00.577517 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:59:07.187190 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:59:07.187148 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:59:34.991888 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:59:34.991847 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:59:44.688811 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:59:44.688772 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 15:59:53.790339 ip-10-0-132-149 kubenswrapper[2560]: I0724 15:59:53.790285 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:00:00.148279 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:00.148246 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn"] Jul 24 16:00:00.152025 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:00.152006 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" Jul 24 16:00:00.154436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:00.154415 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:00:00.158886 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:00.158822 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn"] Jul 24 16:00:00.164127 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:00.164104 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" Jul 24 16:00:00.308387 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:00.308363 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn"] Jul 24 16:00:00.310705 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:00:00.310676 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc85fd8_9564_4abd_b74b_59c3d496e102.slice/crio-d3c90da4008b79206433c235843a3a44c8ba4b0338bf13906b556aabe01e6119 WatchSource:0}: Error finding container d3c90da4008b79206433c235843a3a44c8ba4b0338bf13906b556aabe01e6119: Status 404 returned error can't find the container with id d3c90da4008b79206433c235843a3a44c8ba4b0338bf13906b556aabe01e6119 Jul 24 16:00:00.313029 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:00.313013 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 16:00:01.056317 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:01.056276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" event={"ID":"1cc85fd8-9564-4abd-b74b-59c3d496e102","Type":"ContainerStarted","Data":"d3c90da4008b79206433c235843a3a44c8ba4b0338bf13906b556aabe01e6119"} Jul 24 16:00:03.069276 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:03.069230 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" event={"ID":"1cc85fd8-9564-4abd-b74b-59c3d496e102","Type":"ContainerStarted","Data":"adb673dbd4912fe69fcef888b9ffe16e850969ce7a98e229c07b0ddb42a3a53a"} Jul 24 16:00:03.087316 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:03.087227 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" podStartSLOduration=0.988031044 podStartE2EDuration="3.087205255s" totalImagesPullingTime="2.099174211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:00:03.085000331 +0000 UTC m=+1426.507601386" watchObservedRunningTime="2026-07-24 16:00:03.087205255 +0000 UTC m=+1426.509806284" Jul 24 16:00:04.075465 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:04.075429 2560 generic.go:356] "Generic (PLEG): container finished" podID="1cc85fd8-9564-4abd-b74b-59c3d496e102" containerID="adb673dbd4912fe69fcef888b9ffe16e850969ce7a98e229c07b0ddb42a3a53a" exitCode=0 Jul 24 16:00:04.075939 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:04.075514 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" event={"ID":"1cc85fd8-9564-4abd-b74b-59c3d496e102","Type":"ContainerDied","Data":"adb673dbd4912fe69fcef888b9ffe16e850969ce7a98e229c07b0ddb42a3a53a"} Jul 24 16:00:05.217099 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:05.217075 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" Jul 24 16:00:06.085479 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:06.085448 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" Jul 24 16:00:06.085727 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:06.085440 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-mshkn" event={"ID":"1cc85fd8-9564-4abd-b74b-59c3d496e102","Type":"ContainerDied","Data":"d3c90da4008b79206433c235843a3a44c8ba4b0338bf13906b556aabe01e6119"} Jul 24 16:00:06.085727 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:06.085588 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3c90da4008b79206433c235843a3a44c8ba4b0338bf13906b556aabe01e6119" Jul 24 16:00:10.196369 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:10.196269 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:00:27.693333 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:27.693279 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:00:44.195171 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:44.195134 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:00:58.787557 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:00:58.787518 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:01:09.334751 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:01:09.334712 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:01:19.797026 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:01:19.796988 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:01:32.499691 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:01:32.499655 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:01:45.099797 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:01:45.099703 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:01:55.183572 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:01:55.183537 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:01:59.203161 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:01:59.203128 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:08.297208 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:08.297171 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:02:17.996752 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:17.996706 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:02:18.451786 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:18.451754 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:21.672670 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:21.672638 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:21.771430 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:21.771389 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:22.157689 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:22.157656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:27.864501 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:27.864465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:27.931110 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:27.931082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:28.098236 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.098198 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:02:28.100003 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.099973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:28.379669 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.379637 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh"] Jul 24 16:02:28.380074 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.380058 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc85fd8-9564-4abd-b74b-59c3d496e102" containerName="cleanup" Jul 24 16:02:28.380141 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.380076 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc85fd8-9564-4abd-b74b-59c3d496e102" containerName="cleanup" Jul 24 16:02:28.380177 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.380150 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc85fd8-9564-4abd-b74b-59c3d496e102" containerName="cleanup" Jul 24 16:02:28.383173 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.383157 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.391640 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.391612 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5ce23d5e-openshift-default-dockercfg-zrq2v\"" Jul 24 16:02:28.404431 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.404403 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh"] Jul 24 16:02:28.412820 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.412793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:28.570191 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-credential-socket\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570191 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570203 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c5e467-6588-4b17-a857-da691e9619a7-istiod-ca-cert\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570453 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570289 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-socket\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570453 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-istio-token\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570453 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570404 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-envoy\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570554 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570462 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-data\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570554 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6g9f\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-kube-api-access-f6g9f\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570554 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-certs\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.570660 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.570590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c5e467-6588-4b17-a857-da691e9619a7-istio-podinfo\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671330 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-credential-socket\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671330 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c5e467-6588-4b17-a857-da691e9619a7-istiod-ca-cert\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671330 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-socket\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671617 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-istio-token\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671617 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-envoy\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671617 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-data\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671766 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671641 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6g9f\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-kube-api-access-f6g9f\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671766 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671683 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-certs\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671766 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671709 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-credential-socket\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.671766 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c5e467-6588-4b17-a857-da691e9619a7-istio-podinfo\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.672131 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.671832 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-socket\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.672131 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.672008 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-data\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.672131 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.672022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-certs\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.672428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.672405 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c5e467-6588-4b17-a857-da691e9619a7-istiod-ca-cert\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.673778 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.673760 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-envoy\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.674246 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.674226 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c5e467-6588-4b17-a857-da691e9619a7-istio-podinfo\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.679999 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.679969 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6g9f\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-kube-api-access-f6g9f\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.680139 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.680122 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-istio-token\") pod \"e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.700225 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.700197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:28.710441 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.710415 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:28.847661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.847637 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh"] Jul 24 16:02:28.850393 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:02:28.850357 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c5e467_6588_4b17_a857_da691e9619a7.slice/crio-0bf3aba8feda3111c5bf74d8d20c459400c3a1e1e02d75f2b29e3f9b3cc963c3 WatchSource:0}: Error finding container 0bf3aba8feda3111c5bf74d8d20c459400c3a1e1e02d75f2b29e3f9b3cc963c3: Status 404 returned error can't find the container with id 0bf3aba8feda3111c5bf74d8d20c459400c3a1e1e02d75f2b29e3f9b3cc963c3 Jul 24 16:02:28.852508 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.852467 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:02:28.852617 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.852557 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:02:28.852617 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:28.852603 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:02:29.776821 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:29.776786 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" event={"ID":"d7c5e467-6588-4b17-a857-da691e9619a7","Type":"ContainerStarted","Data":"c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c"} Jul 24 16:02:29.777223 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:29.776842 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" event={"ID":"d7c5e467-6588-4b17-a857-da691e9619a7","Type":"ContainerStarted","Data":"0bf3aba8feda3111c5bf74d8d20c459400c3a1e1e02d75f2b29e3f9b3cc963c3"} Jul 24 16:02:29.801053 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:29.801000 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" podStartSLOduration=1.800978881 podStartE2EDuration="1.800978881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:02:29.799175957 +0000 UTC m=+1573.221777005" watchObservedRunningTime="2026-07-24 16:02:29.800978881 +0000 UTC m=+1573.223579914" Jul 24 16:02:30.700881 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:30.700848 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:30.706170 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:30.706147 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:30.781883 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:30.781846 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:30.782917 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:30.782898 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:31.170950 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:31.170912 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:31.273039 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:31.272997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:31.436079 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:31.435979 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:31.931715 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:31.931667 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:32.026942 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:32.026910 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:32.115171 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:32.115142 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:02:39.299185 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:39.299148 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:02:46.824501 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:46.824449 2560 patch_prober.go:28] interesting pod/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 16:02:46.824942 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:46.824533 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 16:02:47.864795 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:47.864763 2560 generic.go:356] "Generic (PLEG): container finished" podID="d7c5e467-6588-4b17-a857-da691e9619a7" containerID="c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c" exitCode=0 Jul 24 16:02:47.865230 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:47.864846 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" event={"ID":"d7c5e467-6588-4b17-a857-da691e9619a7","Type":"ContainerDied","Data":"c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c"} Jul 24 16:02:47.865273 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:47.865246 2560 scope.go:117] "RemoveContainer" containerID="c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c" Jul 24 16:02:47.866284 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:47.866249 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:02:47.866427 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:47.866356 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:02:47.866427 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:47.866409 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:02:48.875023 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:48.874978 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" event={"ID":"d7c5e467-6588-4b17-a857-da691e9619a7","Type":"ContainerStarted","Data":"520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3"} Jul 24 16:02:49.700357 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:49.700317 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:49.700672 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:49.700649 2560 patch_prober.go:28] interesting pod/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 24 16:02:49.700724 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:49.700703 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 24 16:02:50.701531 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:50.701492 2560 patch_prober.go:28] interesting pod/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 24 16:02:50.701921 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:50.701560 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 24 16:02:51.701131 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:51.701092 2560 patch_prober.go:28] interesting pod/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 24 16:02:51.701333 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:51.701177 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 24 16:02:52.701136 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:52.701096 2560 patch_prober.go:28] interesting pod/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 24 16:02:52.701554 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:52.701166 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 24 16:02:53.704182 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:53.704153 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:53.704565 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:53.704493 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:53.705202 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:53.705185 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:02:56.785733 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:56.785694 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x"] Jul 24 16:02:56.789132 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:56.789108 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" Jul 24 16:02:56.791542 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:56.791519 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:02:56.791648 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:56.791519 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:02:56.797089 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:56.797060 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x"] Jul 24 16:02:56.953928 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:56.953886 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2ba6-ec45-45ce-8599-99ec7f567f0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x\" (UID: \"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" Jul 24 16:02:57.055191 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:57.055100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2ba6-ec45-45ce-8599-99ec7f567f0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x\" (UID: \"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" Jul 24 16:02:57.055794 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:57.055772 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2ba6-ec45-45ce-8599-99ec7f567f0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x\" (UID: \"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" Jul 24 16:02:57.100563 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:57.100529 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" Jul 24 16:02:57.238597 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:57.238562 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x"] Jul 24 16:02:57.240928 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:02:57.240901 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fa2ba6_ec45_45ce_8599_99ec7f567f0f.slice/crio-8e392a5d6db9c395c2ef752f3364fcb7c2e7f2b69572b5702bd91b02cdaf00b4 WatchSource:0}: Error finding container 8e392a5d6db9c395c2ef752f3364fcb7c2e7f2b69572b5702bd91b02cdaf00b4: Status 404 returned error can't find the container with id 8e392a5d6db9c395c2ef752f3364fcb7c2e7f2b69572b5702bd91b02cdaf00b4 Jul 24 16:02:57.922868 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:57.922835 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" event={"ID":"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f","Type":"ContainerStarted","Data":"af842c96e41b5f1eb976ccbff27b98aea1c0f0a4eb2253a79123f11db2e0b11f"} Jul 24 16:02:57.923231 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:57.922887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" event={"ID":"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f","Type":"ContainerStarted","Data":"8e392a5d6db9c395c2ef752f3364fcb7c2e7f2b69572b5702bd91b02cdaf00b4"} Jul 24 16:02:57.942169 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:57.942115 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" podStartSLOduration=1.942099947 podStartE2EDuration="1.942099947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:02:57.940484202 +0000 UTC m=+1601.363085258" watchObservedRunningTime="2026-07-24 16:02:57.942099947 +0000 UTC m=+1601.364701093" Jul 24 16:02:58.928598 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:58.928560 2560 generic.go:356] "Generic (PLEG): container finished" podID="f8fa2ba6-ec45-45ce-8599-99ec7f567f0f" containerID="af842c96e41b5f1eb976ccbff27b98aea1c0f0a4eb2253a79123f11db2e0b11f" exitCode=0 Jul 24 16:02:58.928993 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:02:58.928653 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" event={"ID":"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f","Type":"ContainerDied","Data":"af842c96e41b5f1eb976ccbff27b98aea1c0f0a4eb2253a79123f11db2e0b11f"} Jul 24 16:03:00.066096 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:00.066072 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" Jul 24 16:03:00.183873 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:00.183782 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2ba6-ec45-45ce-8599-99ec7f567f0f-maas-api-service-ca\") pod \"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f\" (UID: \"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f\") " Jul 24 16:03:00.184179 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:00.184156 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8fa2ba6-ec45-45ce-8599-99ec7f567f0f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8fa2ba6-ec45-45ce-8599-99ec7f567f0f" (UID: "f8fa2ba6-ec45-45ce-8599-99ec7f567f0f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:03:00.284780 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:00.284743 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2ba6-ec45-45ce-8599-99ec7f567f0f-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:00.938740 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:00.938712 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" Jul 24 16:03:00.938740 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:00.938726 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x" event={"ID":"f8fa2ba6-ec45-45ce-8599-99ec7f567f0f","Type":"ContainerDied","Data":"8e392a5d6db9c395c2ef752f3364fcb7c2e7f2b69572b5702bd91b02cdaf00b4"} Jul 24 16:03:00.938956 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:00.938776 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e392a5d6db9c395c2ef752f3364fcb7c2e7f2b69572b5702bd91b02cdaf00b4" Jul 24 16:03:07.351523 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:07.351447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:07.451328 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:07.451271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:10.499323 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:10.499263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:03:11.877056 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:11.877025 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x"] Jul 24 16:03:11.877628 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:11.877493 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:11.879638 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:11.879616 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce23d5e-5fp2x"] Jul 24 16:03:12.025742 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:12.025705 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh"] Jul 24 16:03:12.025958 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:12.025935 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" containerID="cri-o://520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3" gracePeriod=30 Jul 24 16:03:12.050618 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:12.050575 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:12.342787 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:12.342747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:12.604675 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:12.604576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:12.704367 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:12.704328 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:12.833481 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:12.833441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:13.141533 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.141497 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8fa2ba6-ec45-45ce-8599-99ec7f567f0f" path="/var/lib/kubelet/pods/f8fa2ba6-ec45-45ce-8599-99ec7f567f0f/volumes" Jul 24 16:03:13.174564 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.174521 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4"] Jul 24 16:03:13.175113 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.175091 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fa2ba6-ec45-45ce-8599-99ec7f567f0f" containerName="revoke-keys" Jul 24 16:03:13.175113 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.175113 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fa2ba6-ec45-45ce-8599-99ec7f567f0f" containerName="revoke-keys" Jul 24 16:03:13.175255 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.175191 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fa2ba6-ec45-45ce-8599-99ec7f567f0f" containerName="revoke-keys" Jul 24 16:03:13.178498 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.178479 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.182616 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.182598 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-09aa8ba8-gw-openshift-default-dockercfg-nr4rc\"" Jul 24 16:03:13.206355 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.204960 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4"] Jul 24 16:03:13.211685 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.211653 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-data\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.211842 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.211801 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ec354d-dcff-488f-a268-7d64ffe1770a-istiod-ca-cert\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.212019 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.212002 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-credential-socket\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.212094 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.212062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-certs\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.212319 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.212269 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-token\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.212463 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.212358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-podinfo\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.212463 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.212433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-envoy\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.212593 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.212462 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-socket\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.212679 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.212655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v7f5\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-kube-api-access-2v7f5\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.229361 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.229285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:13.313980 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.313944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ec354d-dcff-488f-a268-7d64ffe1770a-istiod-ca-cert\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.313992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-credential-socket\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314019 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-certs\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314053 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-token\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-podinfo\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-envoy\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-socket\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v7f5\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-kube-api-access-2v7f5\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314272 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-data\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314649 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314524 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-credential-socket\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314649 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314603 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-socket\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314745 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314675 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-data\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314745 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314713 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-certs\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.314972 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.314955 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ec354d-dcff-488f-a268-7d64ffe1770a-istiod-ca-cert\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.316830 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.316805 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-envoy\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.316931 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.316812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-podinfo\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.328612 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.328584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v7f5\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-kube-api-access-2v7f5\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.329723 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.329697 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-token\") pod \"e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.489623 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.489524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:13.642958 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.642921 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4"] Jul 24 16:03:13.645955 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:03:13.645928 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ec354d_dcff_488f_a268_7d64ffe1770a.slice/crio-6d6e510feae188d8fe3f3bca0ef72f1fe1fd5d84eb91210df522f0770fb10165 WatchSource:0}: Error finding container 6d6e510feae188d8fe3f3bca0ef72f1fe1fd5d84eb91210df522f0770fb10165: Status 404 returned error can't find the container with id 6d6e510feae188d8fe3f3bca0ef72f1fe1fd5d84eb91210df522f0770fb10165 Jul 24 16:03:13.647976 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.647925 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:13.648084 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.648003 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:13.648084 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.648034 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:13.710511 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:13.710480 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:14.006479 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:14.006443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" event={"ID":"22ec354d-dcff-488f-a268-7d64ffe1770a","Type":"ContainerStarted","Data":"6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f"} Jul 24 16:03:14.006642 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:14.006498 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" event={"ID":"22ec354d-dcff-488f-a268-7d64ffe1770a","Type":"ContainerStarted","Data":"6d6e510feae188d8fe3f3bca0ef72f1fe1fd5d84eb91210df522f0770fb10165"} Jul 24 16:03:14.027470 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:14.027417 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" podStartSLOduration=1.027399718 podStartE2EDuration="1.027399718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:03:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:03:14.025989757 +0000 UTC m=+1617.448590802" watchObservedRunningTime="2026-07-24 16:03:14.027399718 +0000 UTC m=+1617.450000759" Jul 24 16:03:14.489711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:14.489677 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:14.494675 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:14.494650 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:15.011984 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:15.011952 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:15.012959 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:15.012936 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:15.514498 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:15.514465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:15.632038 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:15.632005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:15.806527 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:15.806444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:16.172543 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:16.172465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:16.265889 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:16.265851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:17.275357 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.275333 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:03:17.355325 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355192 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c5e467-6588-4b17-a857-da691e9619a7-istiod-ca-cert\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355325 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355249 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-istio-token\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355325 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355285 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-envoy\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355358 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-socket\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355387 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6g9f\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-kube-api-access-f6g9f\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355420 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-certs\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355464 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-data\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355499 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-credential-socket\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355554 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c5e467-6588-4b17-a857-da691e9619a7-istio-podinfo\") pod \"d7c5e467-6588-4b17-a857-da691e9619a7\" (UID: \"d7c5e467-6588-4b17-a857-da691e9619a7\") " Jul 24 16:03:17.355922 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355689 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7c5e467-6588-4b17-a857-da691e9619a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:03:17.355922 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355722 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:03:17.356041 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355931 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:03:17.356041 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.355943 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:03:17.356169 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.356129 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c5e467-6588-4b17-a857-da691e9619a7-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.356225 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.356171 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.356225 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.356187 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.356225 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.356201 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.356672 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.356646 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:03:17.358012 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.357976 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7c5e467-6588-4b17-a857-da691e9619a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:03:17.358130 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.358062 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:03:17.358130 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.358089 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-kube-api-access-f6g9f" (OuterVolumeSpecName: "kube-api-access-f6g9f") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "kube-api-access-f6g9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:03:17.358130 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.358109 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7c5e467-6588-4b17-a857-da691e9619a7" (UID: "d7c5e467-6588-4b17-a857-da691e9619a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:03:17.457016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.456959 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.457016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.457004 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6g9f\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-kube-api-access-f6g9f\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.457016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.457016 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c5e467-6588-4b17-a857-da691e9619a7-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.457016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.457025 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c5e467-6588-4b17-a857-da691e9619a7-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:17.457016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:17.457032 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c5e467-6588-4b17-a857-da691e9619a7-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:18.030691 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.030643 2560 generic.go:356] "Generic (PLEG): container finished" podID="d7c5e467-6588-4b17-a857-da691e9619a7" containerID="520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3" exitCode=0 Jul 24 16:03:18.030691 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.030692 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" event={"ID":"d7c5e467-6588-4b17-a857-da691e9619a7","Type":"ContainerDied","Data":"520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3"} Jul 24 16:03:18.030931 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.030731 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" Jul 24 16:03:18.030931 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.030778 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh" event={"ID":"d7c5e467-6588-4b17-a857-da691e9619a7","Type":"ContainerDied","Data":"0bf3aba8feda3111c5bf74d8d20c459400c3a1e1e02d75f2b29e3f9b3cc963c3"} Jul 24 16:03:18.030931 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.030810 2560 scope.go:117] "RemoveContainer" containerID="520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3" Jul 24 16:03:18.041084 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.041065 2560 scope.go:117] "RemoveContainer" containerID="c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c" Jul 24 16:03:18.049198 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.049174 2560 scope.go:117] "RemoveContainer" containerID="520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3" Jul 24 16:03:18.049576 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:03:18.049556 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3\": container with ID starting with 520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3 not found: ID does not exist" containerID="520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3" Jul 24 16:03:18.049656 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.049585 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3"} err="failed to get container status \"520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3\": rpc error: code = NotFound desc = could not find container \"520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3\": container with ID starting with 520fe46d26f0651949b624a76e147ad3fa79cad378451de952cf09f0d094c2c3 not found: ID does not exist" Jul 24 16:03:18.049656 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.049602 2560 scope.go:117] "RemoveContainer" containerID="c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c" Jul 24 16:03:18.049853 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:03:18.049839 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c\": container with ID starting with c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c not found: ID does not exist" containerID="c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c" Jul 24 16:03:18.049893 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.049858 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c"} err="failed to get container status \"c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c\": rpc error: code = NotFound desc = could not find container \"c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c\": container with ID starting with c798a7a80b2d0d75ddf0e696c6bfd5e332f26e8ff98941b0f8c81829b708092c not found: ID does not exist" Jul 24 16:03:18.053318 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.053276 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh"] Jul 24 16:03:18.056834 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:18.056799 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5ce23d5e-openshift-default-756c6b7848-csqgh"] Jul 24 16:03:19.141080 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:19.141047 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" path="/var/lib/kubelet/pods/d7c5e467-6588-4b17-a857-da691e9619a7/volumes" Jul 24 16:03:22.793521 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:22.793481 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:03:30.643366 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:30.642721 2560 patch_prober.go:28] interesting pod/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": read tcp 10.134.0.2:59124->10.134.0.32:15021: read: connection reset by peer" start-of-body= Jul 24 16:03:30.643366 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:30.642798 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": read tcp 10.134.0.2:59124->10.134.0.32:15021: read: connection reset by peer" Jul 24 16:03:31.107084 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:31.107051 2560 generic.go:356] "Generic (PLEG): container finished" podID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerID="6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f" exitCode=0 Jul 24 16:03:31.107257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:31.107118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" event={"ID":"22ec354d-dcff-488f-a268-7d64ffe1770a","Type":"ContainerDied","Data":"6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f"} Jul 24 16:03:31.107554 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:31.107540 2560 scope.go:117] "RemoveContainer" containerID="6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f" Jul 24 16:03:31.108561 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:31.108533 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:31.108621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:31.108599 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:31.108658 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:31.108630 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:32.114412 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:32.114376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" event={"ID":"22ec354d-dcff-488f-a268-7d64ffe1770a","Type":"ContainerStarted","Data":"67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044"} Jul 24 16:03:32.489784 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:32.489676 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:32.489986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:32.489961 2560 patch_prober.go:28] interesting pod/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 24 16:03:32.490054 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:32.490026 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 24 16:03:33.490825 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:33.490786 2560 patch_prober.go:28] interesting pod/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 24 16:03:33.491224 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:33.490842 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 24 16:03:34.490322 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:34.490267 2560 patch_prober.go:28] interesting pod/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 24 16:03:34.490536 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:34.490373 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 24 16:03:35.490191 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:35.490155 2560 patch_prober.go:28] interesting pod/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 24 16:03:35.490614 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:35.490209 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 24 16:03:36.494097 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:36.494065 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:36.494575 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:36.494341 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:36.495039 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:36.495023 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:03:40.602956 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.602918 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx"] Jul 24 16:03:40.603382 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.603325 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" Jul 24 16:03:40.603382 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.603337 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" Jul 24 16:03:40.603382 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.603359 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" Jul 24 16:03:40.603382 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.603364 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" Jul 24 16:03:40.603573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.603425 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" Jul 24 16:03:40.603573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.603433 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c5e467-6588-4b17-a857-da691e9619a7" containerName="istio-proxy" Jul 24 16:03:40.605514 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.605499 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" Jul 24 16:03:40.608010 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.607968 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:03:40.608139 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.608090 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:03:40.613689 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.613662 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx"] Jul 24 16:03:40.691443 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.691409 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1e1e90-1fe3-4223-9da1-faccb10541f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx\" (UID: \"0a1e1e90-1fe3-4223-9da1-faccb10541f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" Jul 24 16:03:40.791986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.791944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1e1e90-1fe3-4223-9da1-faccb10541f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx\" (UID: \"0a1e1e90-1fe3-4223-9da1-faccb10541f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" Jul 24 16:03:40.792704 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.792684 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1e1e90-1fe3-4223-9da1-faccb10541f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx\" (UID: \"0a1e1e90-1fe3-4223-9da1-faccb10541f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" Jul 24 16:03:40.917459 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:40.917376 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" Jul 24 16:03:41.054235 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:41.054209 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx"] Jul 24 16:03:41.056169 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:03:41.056130 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1e1e90_1fe3_4223_9da1_faccb10541f1.slice/crio-a04e06c759d44be1f39087eb7bb177bafacb01aeb36ad5bf2d49f0432ec8523a WatchSource:0}: Error finding container a04e06c759d44be1f39087eb7bb177bafacb01aeb36ad5bf2d49f0432ec8523a: Status 404 returned error can't find the container with id a04e06c759d44be1f39087eb7bb177bafacb01aeb36ad5bf2d49f0432ec8523a Jul 24 16:03:41.157486 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:41.157453 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" event={"ID":"0a1e1e90-1fe3-4223-9da1-faccb10541f1","Type":"ContainerStarted","Data":"a04e06c759d44be1f39087eb7bb177bafacb01aeb36ad5bf2d49f0432ec8523a"} Jul 24 16:03:42.166780 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:42.166691 2560 generic.go:356] "Generic (PLEG): container finished" podID="0a1e1e90-1fe3-4223-9da1-faccb10541f1" containerID="c8f69ad5439ae4d717866128c6312c66d8657bfee55e0bc487318cd1a2f0c52a" exitCode=0 Jul 24 16:03:42.167127 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:42.166771 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" event={"ID":"0a1e1e90-1fe3-4223-9da1-faccb10541f1","Type":"ContainerDied","Data":"c8f69ad5439ae4d717866128c6312c66d8657bfee55e0bc487318cd1a2f0c52a"} Jul 24 16:03:43.302017 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:43.301988 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" Jul 24 16:03:43.313192 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:43.313171 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1e1e90-1fe3-4223-9da1-faccb10541f1-maas-api-service-ca\") pod \"0a1e1e90-1fe3-4223-9da1-faccb10541f1\" (UID: \"0a1e1e90-1fe3-4223-9da1-faccb10541f1\") " Jul 24 16:03:43.313507 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:43.313484 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a1e1e90-1fe3-4223-9da1-faccb10541f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a1e1e90-1fe3-4223-9da1-faccb10541f1" (UID: "0a1e1e90-1fe3-4223-9da1-faccb10541f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:03:43.414615 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:43.414572 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a1e1e90-1fe3-4223-9da1-faccb10541f1-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:03:44.179517 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:44.179488 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" Jul 24 16:03:44.179689 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:44.179523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx" event={"ID":"0a1e1e90-1fe3-4223-9da1-faccb10541f1","Type":"ContainerDied","Data":"a04e06c759d44be1f39087eb7bb177bafacb01aeb36ad5bf2d49f0432ec8523a"} Jul 24 16:03:44.179689 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:44.179565 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a04e06c759d44be1f39087eb7bb177bafacb01aeb36ad5bf2d49f0432ec8523a" Jul 24 16:03:51.063903 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:51.063849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:52.827819 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:52.827783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:54.595665 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:54.595627 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:03:55.674370 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:55.674334 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:55.674797 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:55.674777 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx"] Jul 24 16:03:55.678050 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:55.678028 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-09aa8ba8-7h7zx"] Jul 24 16:03:55.825274 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:55.825243 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4"] Jul 24 16:03:55.825517 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:55.825492 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" containerID="cri-o://67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044" gracePeriod=30 Jul 24 16:03:55.850226 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:55.850191 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:56.101818 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.101735 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:56.361273 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.361186 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:56.659273 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.659193 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99"] Jul 24 16:03:56.659671 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.659655 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a1e1e90-1fe3-4223-9da1-faccb10541f1" containerName="revoke-keys" Jul 24 16:03:56.659719 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.659675 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a1e1e90-1fe3-4223-9da1-faccb10541f1" containerName="revoke-keys" Jul 24 16:03:56.659770 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.659759 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a1e1e90-1fe3-4223-9da1-faccb10541f1" containerName="revoke-keys" Jul 24 16:03:56.661873 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.661856 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.664593 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.664570 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-510804df-openshift-default-dockercfg-v5hq8\"" Jul 24 16:03:56.677273 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.677237 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99"] Jul 24 16:03:56.696655 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.696626 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:56.743190 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-credential-socket\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-data\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743335 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-token\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743373 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-envoy\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743407 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-socket\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743654 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743442 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-podinfo\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743654 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-certs\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743654 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743519 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8rrs\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-kube-api-access-f8rrs\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.743654 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.743571 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3880b5cd-433b-4b9d-918b-6a1200a075ce-istiod-ca-cert\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845201 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845163 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-credential-socket\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-data\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-token\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-envoy\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-socket\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845693 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845463 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-podinfo\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845693 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-certs\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845693 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845539 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8rrs\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-kube-api-access-f8rrs\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845693 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3880b5cd-433b-4b9d-918b-6a1200a075ce-istiod-ca-cert\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.845693 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845663 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-credential-socket\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.846039 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845712 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-data\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.846039 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845776 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-socket\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.846039 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.845865 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-certs\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.846382 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.846361 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3880b5cd-433b-4b9d-918b-6a1200a075ce-istiod-ca-cert\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.847825 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.847808 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-envoy\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.848443 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.848427 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-podinfo\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.853185 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.853161 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-token\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.853717 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.853700 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8rrs\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-kube-api-access-f8rrs\") pod \"e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:56.865260 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.865235 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:56.972532 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:56.972435 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:57.114846 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.114820 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99"] Jul 24 16:03:57.116613 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:03:57.116579 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3880b5cd_433b_4b9d_918b_6a1200a075ce.slice/crio-d30db90b7d7b7377cea22b29b704d3f695cde92b5a3421c7c58a2c71f1685cf4 WatchSource:0}: Error finding container d30db90b7d7b7377cea22b29b704d3f695cde92b5a3421c7c58a2c71f1685cf4: Status 404 returned error can't find the container with id d30db90b7d7b7377cea22b29b704d3f695cde92b5a3421c7c58a2c71f1685cf4 Jul 24 16:03:57.118730 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.118698 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:57.118814 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.118763 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:57.118814 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.118789 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:03:57.142032 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.142004 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a1e1e90-1fe3-4223-9da1-faccb10541f1" path="/var/lib/kubelet/pods/0a1e1e90-1fe3-4223-9da1-faccb10541f1/volumes" Jul 24 16:03:57.245316 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.245198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" event={"ID":"3880b5cd-433b-4b9d-918b-6a1200a075ce","Type":"ContainerStarted","Data":"8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20"} Jul 24 16:03:57.245316 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.245264 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" event={"ID":"3880b5cd-433b-4b9d-918b-6a1200a075ce","Type":"ContainerStarted","Data":"d30db90b7d7b7377cea22b29b704d3f695cde92b5a3421c7c58a2c71f1685cf4"} Jul 24 16:03:57.266831 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.266780 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" podStartSLOduration=1.2667622729999999 podStartE2EDuration="1.266762273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:03:57.264506781 +0000 UTC m=+1660.687107862" watchObservedRunningTime="2026-07-24 16:03:57.266762273 +0000 UTC m=+1660.689363301" Jul 24 16:03:57.973118 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:57.973080 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:58.613490 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:58.613441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:58.708759 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:58.708722 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:58.842257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:58.842218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:58.978259 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:58.978177 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:59.067574 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:59.067538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:59.255515 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:59.255434 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:59.256255 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:59.256231 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:03:59.395160 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:59.395127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:03:59.499605 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:03:59.499576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:01.091582 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.091526 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:04:01.186859 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.186820 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ec354d-dcff-488f-a268-7d64ffe1770a-istiod-ca-cert\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.186859 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.186865 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-token\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187128 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.186891 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-data\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187128 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.186909 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-envoy\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187128 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.186955 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-credential-socket\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187128 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.186995 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-socket\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187128 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187061 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-certs\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187128 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187086 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-podinfo\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187451 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187180 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v7f5\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-kube-api-access-2v7f5\") pod \"22ec354d-dcff-488f-a268-7d64ffe1770a\" (UID: \"22ec354d-dcff-488f-a268-7d64ffe1770a\") " Jul 24 16:04:01.187451 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187271 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:01.187451 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187422 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:01.187451 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187445 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ec354d-dcff-488f-a268-7d64ffe1770a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:04:01.187665 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187627 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ec354d-dcff-488f-a268-7d64ffe1770a-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.187665 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187639 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:01.187665 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187663 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.187812 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.187679 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.188161 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.188120 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-data" (OuterVolumeSpecName: "istio-data") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:01.189558 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.189533 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-token" (OuterVolumeSpecName: "istio-token") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:04:01.190022 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.190002 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:04:01.190100 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.190055 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-kube-api-access-2v7f5" (OuterVolumeSpecName: "kube-api-access-2v7f5") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "kube-api-access-2v7f5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:04:01.190100 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.190054 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22ec354d-dcff-488f-a268-7d64ffe1770a" (UID: "22ec354d-dcff-488f-a268-7d64ffe1770a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:01.268124 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.268085 2560 generic.go:356] "Generic (PLEG): container finished" podID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerID="67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044" exitCode=0 Jul 24 16:04:01.268322 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.268155 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" Jul 24 16:04:01.268322 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.268172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" event={"ID":"22ec354d-dcff-488f-a268-7d64ffe1770a","Type":"ContainerDied","Data":"67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044"} Jul 24 16:04:01.268322 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.268268 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4" event={"ID":"22ec354d-dcff-488f-a268-7d64ffe1770a","Type":"ContainerDied","Data":"6d6e510feae188d8fe3f3bca0ef72f1fe1fd5d84eb91210df522f0770fb10165"} Jul 24 16:04:01.268322 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.268296 2560 scope.go:117] "RemoveContainer" containerID="67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044" Jul 24 16:04:01.280784 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.280761 2560 scope.go:117] "RemoveContainer" containerID="6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f" Jul 24 16:04:01.288454 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.288428 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.288454 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.288453 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.288622 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.288465 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v7f5\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-kube-api-access-2v7f5\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.288622 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.288474 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.288622 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.288482 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.288622 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.288491 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ec354d-dcff-488f-a268-7d64ffe1770a-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:01.290172 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.290150 2560 scope.go:117] "RemoveContainer" containerID="67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044" Jul 24 16:04:01.290854 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:04:01.290818 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044\": container with ID starting with 67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044 not found: ID does not exist" containerID="67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044" Jul 24 16:04:01.291037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.290859 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044"} err="failed to get container status \"67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044\": rpc error: code = NotFound desc = could not find container \"67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044\": container with ID starting with 67dd45ff3a8ab0cea8ff83119f1425b2e820cb8a3bfca72085ae5273c76e3044 not found: ID does not exist" Jul 24 16:04:01.291037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.290888 2560 scope.go:117] "RemoveContainer" containerID="6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f" Jul 24 16:04:01.291336 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:04:01.291280 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f\": container with ID starting with 6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f not found: ID does not exist" containerID="6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f" Jul 24 16:04:01.291489 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.291341 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f"} err="failed to get container status \"6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f\": rpc error: code = NotFound desc = could not find container \"6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f\": container with ID starting with 6ff32a4a5bd1a5c045e1b000cb6526b4003d5c76086e441095d19505beb79c6f not found: ID does not exist" Jul 24 16:04:01.293193 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.293169 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4"] Jul 24 16:04:01.299387 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:01.299358 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-09aa8ba8-gw-openshift-default-5977548476-f6mm4"] Jul 24 16:04:03.141533 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:03.141499 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" path="/var/lib/kubelet/pods/22ec354d-dcff-488f-a268-7d64ffe1770a/volumes" Jul 24 16:04:06.192973 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:06.192931 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:04:14.256820 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.256734 2560 patch_prober.go:28] interesting pod/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 16:04:14.256820 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.256802 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 16:04:14.335661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.335566 2560 generic.go:356] "Generic (PLEG): container finished" podID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerID="8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20" exitCode=0 Jul 24 16:04:14.335661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.335614 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" event={"ID":"3880b5cd-433b-4b9d-918b-6a1200a075ce","Type":"ContainerDied","Data":"8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20"} Jul 24 16:04:14.336008 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.335990 2560 scope.go:117] "RemoveContainer" containerID="8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20" Jul 24 16:04:14.337135 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.337097 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:14.337227 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.337211 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:14.337274 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:14.337259 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:15.349005 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:15.348960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" event={"ID":"3880b5cd-433b-4b9d-918b-6a1200a075ce","Type":"ContainerStarted","Data":"69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962"} Jul 24 16:04:15.973921 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:15.973878 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:04:15.974135 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:15.974010 2560 patch_prober.go:28] interesting pod/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 16:04:15.974135 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:15.974071 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 16:04:16.973466 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:16.973425 2560 patch_prober.go:28] interesting pod/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 16:04:16.973959 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:16.973487 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 16:04:17.972950 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:17.972906 2560 patch_prober.go:28] interesting pod/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 16:04:17.973163 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:17.973003 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 16:04:18.973701 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:18.973656 2560 patch_prober.go:28] interesting pod/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 16:04:18.974232 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:18.973722 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 16:04:19.976556 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:19.976522 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:04:19.977006 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:19.976784 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:04:19.977450 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:19.977432 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:04:24.513587 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.513552 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd"] Jul 24 16:04:24.513979 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.513928 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" Jul 24 16:04:24.513979 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.513939 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" Jul 24 16:04:24.513979 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.513955 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" Jul 24 16:04:24.513979 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.513961 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" Jul 24 16:04:24.514110 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.514038 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" Jul 24 16:04:24.514110 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.514046 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="22ec354d-dcff-488f-a268-7d64ffe1770a" containerName="istio-proxy" Jul 24 16:04:24.518494 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.518476 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" Jul 24 16:04:24.521184 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.521150 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:04:24.521337 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.521170 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:04:24.524843 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.524817 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd"] Jul 24 16:04:24.642718 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.642681 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73108fbf-c834-4238-95a6-fc485b0efbac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-510804df-4fqxd\" (UID: \"73108fbf-c834-4238-95a6-fc485b0efbac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" Jul 24 16:04:24.743532 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.743485 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73108fbf-c834-4238-95a6-fc485b0efbac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-510804df-4fqxd\" (UID: \"73108fbf-c834-4238-95a6-fc485b0efbac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" Jul 24 16:04:24.744131 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.744109 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73108fbf-c834-4238-95a6-fc485b0efbac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-510804df-4fqxd\" (UID: \"73108fbf-c834-4238-95a6-fc485b0efbac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" Jul 24 16:04:24.830193 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.830099 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" Jul 24 16:04:24.967012 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:24.966965 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd"] Jul 24 16:04:24.972422 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:04:24.972387 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73108fbf_c834_4238_95a6_fc485b0efbac.slice/crio-b45546cf8855fe8520debeba38b917fa8aa94a707bdb3eb300d62b6d39d9861d WatchSource:0}: Error finding container b45546cf8855fe8520debeba38b917fa8aa94a707bdb3eb300d62b6d39d9861d: Status 404 returned error can't find the container with id b45546cf8855fe8520debeba38b917fa8aa94a707bdb3eb300d62b6d39d9861d Jul 24 16:04:25.403127 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:25.403085 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" event={"ID":"73108fbf-c834-4238-95a6-fc485b0efbac","Type":"ContainerStarted","Data":"a1a51d495ab2b87e6dc6a0a111949a7e91a0c78f9b9f9d6ebe249240532c97d0"} Jul 24 16:04:25.403373 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:25.403149 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" event={"ID":"73108fbf-c834-4238-95a6-fc485b0efbac","Type":"ContainerStarted","Data":"b45546cf8855fe8520debeba38b917fa8aa94a707bdb3eb300d62b6d39d9861d"} Jul 24 16:04:25.419641 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:25.419595 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" podStartSLOduration=1.419580102 podStartE2EDuration="1.419580102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:04:25.417339186 +0000 UTC m=+1688.839940228" watchObservedRunningTime="2026-07-24 16:04:25.419580102 +0000 UTC m=+1688.842181142" Jul 24 16:04:26.409869 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:26.409828 2560 generic.go:356] "Generic (PLEG): container finished" podID="73108fbf-c834-4238-95a6-fc485b0efbac" containerID="a1a51d495ab2b87e6dc6a0a111949a7e91a0c78f9b9f9d6ebe249240532c97d0" exitCode=0 Jul 24 16:04:26.410244 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:26.409912 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" event={"ID":"73108fbf-c834-4238-95a6-fc485b0efbac","Type":"ContainerDied","Data":"a1a51d495ab2b87e6dc6a0a111949a7e91a0c78f9b9f9d6ebe249240532c97d0"} Jul 24 16:04:27.556484 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:27.556456 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" Jul 24 16:04:27.673389 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:27.673264 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73108fbf-c834-4238-95a6-fc485b0efbac-maas-api-service-ca\") pod \"73108fbf-c834-4238-95a6-fc485b0efbac\" (UID: \"73108fbf-c834-4238-95a6-fc485b0efbac\") " Jul 24 16:04:27.673621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:27.673597 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73108fbf-c834-4238-95a6-fc485b0efbac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73108fbf-c834-4238-95a6-fc485b0efbac" (UID: "73108fbf-c834-4238-95a6-fc485b0efbac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:04:27.774041 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:27.774004 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73108fbf-c834-4238-95a6-fc485b0efbac-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:28.420996 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:28.420962 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" Jul 24 16:04:28.420996 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:28.420978 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd" event={"ID":"73108fbf-c834-4238-95a6-fc485b0efbac","Type":"ContainerDied","Data":"b45546cf8855fe8520debeba38b917fa8aa94a707bdb3eb300d62b6d39d9861d"} Jul 24 16:04:28.421232 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:28.421028 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b45546cf8855fe8520debeba38b917fa8aa94a707bdb3eb300d62b6d39d9861d" Jul 24 16:04:35.009016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:35.008979 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:35.109419 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:35.109384 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:35.166997 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:35.166960 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:37.093065 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:37.092989 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:04:39.580684 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:39.580647 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd"] Jul 24 16:04:39.583690 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:39.583661 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-510804df-4fqxd"] Jul 24 16:04:39.584831 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:39.584808 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:40.845712 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:40.845676 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99"] Jul 24 16:04:40.846148 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:40.845911 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" containerID="cri-o://69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962" gracePeriod=30 Jul 24 16:04:41.012568 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.012531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:41.112575 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.112473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:41.140596 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.140563 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73108fbf-c834-4238-95a6-fc485b0efbac" path="/var/lib/kubelet/pods/73108fbf-c834-4238-95a6-fc485b0efbac/volumes" Jul 24 16:04:41.346891 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.346855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:41.659751 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.659707 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf"] Jul 24 16:04:41.660588 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.660562 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73108fbf-c834-4238-95a6-fc485b0efbac" containerName="revoke-keys" Jul 24 16:04:41.660750 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.660728 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="73108fbf-c834-4238-95a6-fc485b0efbac" containerName="revoke-keys" Jul 24 16:04:41.660891 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.660866 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="73108fbf-c834-4238-95a6-fc485b0efbac" containerName="revoke-keys" Jul 24 16:04:41.665447 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.665426 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.668028 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.668008 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-977c6dda-openshift-default-dockercfg-8hxh9\"" Jul 24 16:04:41.675161 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.675138 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf"] Jul 24 16:04:41.695396 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.695362 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:41.812886 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.812842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c0db05-f778-441a-a3f7-18acd7a245c1-istiod-ca-cert\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.812886 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.812889 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-certs\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.813137 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.812918 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-podinfo\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.813137 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.812978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-credential-socket\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.813137 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.813010 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-socket\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.813137 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.813062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cwns\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-kube-api-access-9cwns\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.813137 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.813127 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-data\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.813384 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.813149 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-envoy\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.813384 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.813180 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-token\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.914545 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cwns\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-kube-api-access-9cwns\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.914545 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-data\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.914545 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-envoy\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-token\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c0db05-f778-441a-a3f7-18acd7a245c1-istiod-ca-cert\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914616 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-certs\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-podinfo\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-credential-socket\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.914846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-socket\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.915018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-data\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.915066 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-certs\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915168 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.915145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-credential-socket\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915569 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.915225 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-socket\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.915681 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.915658 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c0db05-f778-441a-a3f7-18acd7a245c1-istiod-ca-cert\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.917135 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.917114 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-envoy\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.917406 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.917388 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-podinfo\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.923013 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.922984 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-token\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.923103 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.923056 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cwns\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-kube-api-access-9cwns\") pod \"e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:41.978110 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:41.978070 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:42.122784 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.122637 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf"] Jul 24 16:04:42.125929 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:04:42.125902 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c0db05_f778_441a_a3f7_18acd7a245c1.slice/crio-c01437d27cba4cd1bc4a81c236a0f47d165f18a4d4befb36cfc01e7f7be6f223 WatchSource:0}: Error finding container c01437d27cba4cd1bc4a81c236a0f47d165f18a4d4befb36cfc01e7f7be6f223: Status 404 returned error can't find the container with id c01437d27cba4cd1bc4a81c236a0f47d165f18a4d4befb36cfc01e7f7be6f223 Jul 24 16:04:42.128101 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.128064 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:42.128177 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.128131 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:42.128177 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.128158 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:42.365037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.364989 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:42.491918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.491805 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" event={"ID":"13c0db05-f778-441a-a3f7-18acd7a245c1","Type":"ContainerStarted","Data":"80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b"} Jul 24 16:04:42.491918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.491872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" event={"ID":"13c0db05-f778-441a-a3f7-18acd7a245c1","Type":"ContainerStarted","Data":"c01437d27cba4cd1bc4a81c236a0f47d165f18a4d4befb36cfc01e7f7be6f223"} Jul 24 16:04:42.527755 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.527688 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" podStartSLOduration=1.527669371 podStartE2EDuration="1.527669371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:04:42.524438064 +0000 UTC m=+1705.947039113" watchObservedRunningTime="2026-07-24 16:04:42.527669371 +0000 UTC m=+1705.950270400" Jul 24 16:04:42.978489 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:42.978446 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:43.880277 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:43.880229 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:43.984182 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:43.984154 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:43.992424 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:43.992391 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:44.064280 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:44.064245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:44.501659 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:44.501626 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:44.502483 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:44.502462 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:44.606912 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:44.606882 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:44.673187 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:44.673152 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:04:46.107880 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.107803 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:04:46.256241 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256208 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8rrs\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-kube-api-access-f8rrs\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256463 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256325 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3880b5cd-433b-4b9d-918b-6a1200a075ce-istiod-ca-cert\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256463 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256371 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-certs\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256463 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256418 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-data\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256463 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256443 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-socket\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256687 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256463 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-token\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256687 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256500 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-podinfo\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256687 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256530 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-credential-socket\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256828 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256769 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3880b5cd-433b-4b9d-918b-6a1200a075ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:04:46.256828 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256780 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:46.256964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256859 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-envoy\") pod \"3880b5cd-433b-4b9d-918b-6a1200a075ce\" (UID: \"3880b5cd-433b-4b9d-918b-6a1200a075ce\") " Jul 24 16:04:46.256964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.256886 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:46.257196 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.257052 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:46.257676 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.257646 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:46.258077 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.258050 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.258183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.258101 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3880b5cd-433b-4b9d-918b-6a1200a075ce-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.258395 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.258373 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.258742 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.258719 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.258898 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.258874 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.259005 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.258937 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-kube-api-access-f8rrs" (OuterVolumeSpecName: "kube-api-access-f8rrs") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "kube-api-access-f8rrs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:04:46.259220 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.259200 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:04:46.259281 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.259226 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:04:46.259372 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.259288 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3880b5cd-433b-4b9d-918b-6a1200a075ce" (UID: "3880b5cd-433b-4b9d-918b-6a1200a075ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:04:46.360407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.360294 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.360407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.360348 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.360407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.360357 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3880b5cd-433b-4b9d-918b-6a1200a075ce-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.360407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.360368 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8rrs\" (UniqueName: \"kubernetes.io/projected/3880b5cd-433b-4b9d-918b-6a1200a075ce-kube-api-access-f8rrs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:04:46.512238 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.512204 2560 generic.go:356] "Generic (PLEG): container finished" podID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerID="69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962" exitCode=0 Jul 24 16:04:46.512442 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.512251 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" Jul 24 16:04:46.512442 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.512312 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" event={"ID":"3880b5cd-433b-4b9d-918b-6a1200a075ce","Type":"ContainerDied","Data":"69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962"} Jul 24 16:04:46.512442 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.512362 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99" event={"ID":"3880b5cd-433b-4b9d-918b-6a1200a075ce","Type":"ContainerDied","Data":"d30db90b7d7b7377cea22b29b704d3f695cde92b5a3421c7c58a2c71f1685cf4"} Jul 24 16:04:46.512442 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.512389 2560 scope.go:117] "RemoveContainer" containerID="69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962" Jul 24 16:04:46.521737 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.521720 2560 scope.go:117] "RemoveContainer" containerID="8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20" Jul 24 16:04:46.529736 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.529716 2560 scope.go:117] "RemoveContainer" containerID="69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962" Jul 24 16:04:46.530016 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:04:46.529998 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962\": container with ID starting with 69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962 not found: ID does not exist" containerID="69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962" Jul 24 16:04:46.530086 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.530028 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962"} err="failed to get container status \"69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962\": rpc error: code = NotFound desc = could not find container \"69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962\": container with ID starting with 69b9e69dea0a19e2d5770d847a7463ee04fff03e303a72e96177b32245cb4962 not found: ID does not exist" Jul 24 16:04:46.530086 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.530053 2560 scope.go:117] "RemoveContainer" containerID="8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20" Jul 24 16:04:46.530879 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:04:46.530600 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20\": container with ID starting with 8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20 not found: ID does not exist" containerID="8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20" Jul 24 16:04:46.530879 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.530681 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20"} err="failed to get container status \"8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20\": rpc error: code = NotFound desc = could not find container \"8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20\": container with ID starting with 8e86ef4c25f229d6881dc6c7b312749e73e6ea82f429e0905a704877abcf1b20 not found: ID does not exist" Jul 24 16:04:46.532728 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.532707 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99"] Jul 24 16:04:46.538438 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:46.538418 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-510804df-openshift-default-67dd4c8bd-2jf99"] Jul 24 16:04:47.141281 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:47.141248 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" path="/var/lib/kubelet/pods/3880b5cd-433b-4b9d-918b-6a1200a075ce/volumes" Jul 24 16:04:47.594381 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:47.594341 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:04:55.562109 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:55.562070 2560 generic.go:356] "Generic (PLEG): container finished" podID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerID="80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b" exitCode=0 Jul 24 16:04:55.562672 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:55.562142 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" event={"ID":"13c0db05-f778-441a-a3f7-18acd7a245c1","Type":"ContainerDied","Data":"80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b"} Jul 24 16:04:55.562672 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:55.562565 2560 scope.go:117] "RemoveContainer" containerID="80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b" Jul 24 16:04:55.563596 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:55.563569 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:55.563666 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:55.563641 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:55.563727 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:55.563671 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:04:56.573508 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:56.573466 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" event={"ID":"13c0db05-f778-441a-a3f7-18acd7a245c1","Type":"ContainerStarted","Data":"ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece"} Jul 24 16:04:56.988668 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:56.988519 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:56.989413 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:56.989389 2560 patch_prober.go:28] interesting pod/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 16:04:56.989538 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:56.989444 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 16:04:57.979934 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:57.979890 2560 patch_prober.go:28] interesting pod/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 16:04:57.980452 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:57.979956 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 16:04:58.979351 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:58.979274 2560 patch_prober.go:28] interesting pod/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 16:04:58.979573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:58.979430 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 16:04:59.502063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:59.502021 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:04:59.978956 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:59.978919 2560 patch_prober.go:28] interesting pod/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 24 16:04:59.979276 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:04:59.979246 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 24 16:05:00.982505 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:00.982465 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:05:00.983498 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:00.983478 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:05:02.241580 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:02.241544 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:02.304337 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:02.304284 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:08.835884 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.835834 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj"] Jul 24 16:05:08.836266 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.836240 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" Jul 24 16:05:08.836266 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.836251 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" Jul 24 16:05:08.836361 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.836343 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" Jul 24 16:05:08.839782 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.839752 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" Jul 24 16:05:08.842007 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.841984 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:05:08.842115 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.842008 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:05:08.845938 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.845912 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj"] Jul 24 16:05:08.997658 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:08.997618 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afea36d4-f337-4d9c-8bd7-635b257603d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj\" (UID: \"afea36d4-f337-4d9c-8bd7-635b257603d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" Jul 24 16:05:09.099257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:09.099160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afea36d4-f337-4d9c-8bd7-635b257603d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj\" (UID: \"afea36d4-f337-4d9c-8bd7-635b257603d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" Jul 24 16:05:09.099790 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:09.099772 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afea36d4-f337-4d9c-8bd7-635b257603d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj\" (UID: \"afea36d4-f337-4d9c-8bd7-635b257603d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" Jul 24 16:05:09.151403 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:09.151359 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" Jul 24 16:05:09.282409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:09.282384 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj"] Jul 24 16:05:09.284674 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:05:09.284645 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafea36d4_f337_4d9c_8bd7_635b257603d8.slice/crio-e295f66f2ea8de861cd702ac44cedf85b0915bf85d2c27cf062df9e39b272119 WatchSource:0}: Error finding container e295f66f2ea8de861cd702ac44cedf85b0915bf85d2c27cf062df9e39b272119: Status 404 returned error can't find the container with id e295f66f2ea8de861cd702ac44cedf85b0915bf85d2c27cf062df9e39b272119 Jul 24 16:05:09.645241 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:09.645133 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" event={"ID":"afea36d4-f337-4d9c-8bd7-635b257603d8","Type":"ContainerStarted","Data":"9f555f047b38b2252aa74cc4717ad8d73d6b7273ab9ce018223658e9d229478d"} Jul 24 16:05:09.645241 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:09.645189 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" event={"ID":"afea36d4-f337-4d9c-8bd7-635b257603d8","Type":"ContainerStarted","Data":"e295f66f2ea8de861cd702ac44cedf85b0915bf85d2c27cf062df9e39b272119"} Jul 24 16:05:09.660213 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:09.660167 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" podStartSLOduration=1.660152681 podStartE2EDuration="1.660152681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:05:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:05:09.658434584 +0000 UTC m=+1733.081035634" watchObservedRunningTime="2026-07-24 16:05:09.660152681 +0000 UTC m=+1733.082753719" Jul 24 16:05:10.651986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:10.651955 2560 generic.go:356] "Generic (PLEG): container finished" podID="afea36d4-f337-4d9c-8bd7-635b257603d8" containerID="9f555f047b38b2252aa74cc4717ad8d73d6b7273ab9ce018223658e9d229478d" exitCode=0 Jul 24 16:05:10.652394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:10.652027 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" event={"ID":"afea36d4-f337-4d9c-8bd7-635b257603d8","Type":"ContainerDied","Data":"9f555f047b38b2252aa74cc4717ad8d73d6b7273ab9ce018223658e9d229478d"} Jul 24 16:05:11.788669 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:11.788644 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" Jul 24 16:05:11.929166 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:11.929076 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afea36d4-f337-4d9c-8bd7-635b257603d8-maas-api-service-ca\") pod \"afea36d4-f337-4d9c-8bd7-635b257603d8\" (UID: \"afea36d4-f337-4d9c-8bd7-635b257603d8\") " Jul 24 16:05:11.929445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:11.929422 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afea36d4-f337-4d9c-8bd7-635b257603d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "afea36d4-f337-4d9c-8bd7-635b257603d8" (UID: "afea36d4-f337-4d9c-8bd7-635b257603d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:05:12.030502 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:12.030461 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afea36d4-f337-4d9c-8bd7-635b257603d8-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:12.664279 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:12.664246 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" event={"ID":"afea36d4-f337-4d9c-8bd7-635b257603d8","Type":"ContainerDied","Data":"e295f66f2ea8de861cd702ac44cedf85b0915bf85d2c27cf062df9e39b272119"} Jul 24 16:05:12.664279 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:12.664277 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj" Jul 24 16:05:12.664519 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:12.664294 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e295f66f2ea8de861cd702ac44cedf85b0915bf85d2c27cf062df9e39b272119" Jul 24 16:05:19.419245 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:19.419204 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:19.520469 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:19.520436 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:19.622597 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:19.622557 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:22.689904 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:22.689862 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:05:23.907591 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:23.907552 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:23.913550 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:23.913517 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj"] Jul 24 16:05:23.916062 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:23.916026 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-977c6dda-2xmjj"] Jul 24 16:05:24.051121 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:24.051087 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf"] Jul 24 16:05:24.051405 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:24.051376 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" containerID="cri-o://ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece" gracePeriod=30 Jul 24 16:05:24.103512 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:24.103477 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:24.349559 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:24.349523 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:24.857980 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:24.857935 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:24.964459 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:24.964419 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:25.058270 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:25.058231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:25.142050 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:25.141951 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afea36d4-f337-4d9c-8bd7-635b257603d8" path="/var/lib/kubelet/pods/afea36d4-f337-4d9c-8bd7-635b257603d8/volumes" Jul 24 16:05:26.670521 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:26.670482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:29.309276 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.309251 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:05:29.384696 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384609 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c0db05-f778-441a-a3f7-18acd7a245c1-istiod-ca-cert\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384696 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384655 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cwns\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-kube-api-access-9cwns\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384696 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384680 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-podinfo\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384733 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-envoy\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384760 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-token\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384783 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-data\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384834 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-certs\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384861 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-credential-socket\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.384964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.384888 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-socket\") pod \"13c0db05-f778-441a-a3f7-18acd7a245c1\" (UID: \"13c0db05-f778-441a-a3f7-18acd7a245c1\") " Jul 24 16:05:29.385276 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.385019 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13c0db05-f778-441a-a3f7-18acd7a245c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:05:29.385276 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.385213 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c0db05-f778-441a-a3f7-18acd7a245c1-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.385276 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.385234 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:05:29.385276 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.385246 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:05:29.385527 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.385393 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:05:29.385876 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.385852 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:05:29.387062 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.387034 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:05:29.387166 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.387142 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-kube-api-access-9cwns" (OuterVolumeSpecName: "kube-api-access-9cwns") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "kube-api-access-9cwns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:05:29.387275 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.387250 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:05:29.387360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.387259 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13c0db05-f778-441a-a3f7-18acd7a245c1" (UID: "13c0db05-f778-441a-a3f7-18acd7a245c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:05:29.486189 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486153 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cwns\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-kube-api-access-9cwns\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.486189 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486181 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.486189 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486190 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.486189 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486199 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.486497 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486208 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.486497 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486215 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.486497 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486222 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.486497 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.486231 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c0db05-f778-441a-a3f7-18acd7a245c1-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:05:29.754212 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.754117 2560 generic.go:356] "Generic (PLEG): container finished" podID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerID="ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece" exitCode=0 Jul 24 16:05:29.754212 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.754167 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" Jul 24 16:05:29.754439 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.754201 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" event={"ID":"13c0db05-f778-441a-a3f7-18acd7a245c1","Type":"ContainerDied","Data":"ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece"} Jul 24 16:05:29.754439 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.754263 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf" event={"ID":"13c0db05-f778-441a-a3f7-18acd7a245c1","Type":"ContainerDied","Data":"c01437d27cba4cd1bc4a81c236a0f47d165f18a4d4befb36cfc01e7f7be6f223"} Jul 24 16:05:29.754439 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.754287 2560 scope.go:117] "RemoveContainer" containerID="ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece" Jul 24 16:05:29.763861 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.763846 2560 scope.go:117] "RemoveContainer" containerID="80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b" Jul 24 16:05:29.771944 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.771926 2560 scope.go:117] "RemoveContainer" containerID="ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece" Jul 24 16:05:29.772212 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:05:29.772192 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece\": container with ID starting with ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece not found: ID does not exist" containerID="ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece" Jul 24 16:05:29.772258 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.772227 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece"} err="failed to get container status \"ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece\": rpc error: code = NotFound desc = could not find container \"ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece\": container with ID starting with ffa88f363804c082877e8494bae59169f04c8276c86f734ca87e73c39a0a9ece not found: ID does not exist" Jul 24 16:05:29.772258 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.772245 2560 scope.go:117] "RemoveContainer" containerID="80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b" Jul 24 16:05:29.772499 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:05:29.772483 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b\": container with ID starting with 80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b not found: ID does not exist" containerID="80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b" Jul 24 16:05:29.772554 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.772502 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b"} err="failed to get container status \"80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b\": rpc error: code = NotFound desc = could not find container \"80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b\": container with ID starting with 80a3ec2b0dd49386563506b6a41cc0c27fbb154214d4df5f728c3899f3738d4b not found: ID does not exist" Jul 24 16:05:29.775638 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.775612 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf"] Jul 24 16:05:29.781495 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:29.781472 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-977c6dda-openshift-default-bc768f84-hjlhf"] Jul 24 16:05:31.141547 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:31.141516 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" path="/var/lib/kubelet/pods/13c0db05-f778-441a-a3f7-18acd7a245c1/volumes" Jul 24 16:05:32.267138 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:32.267103 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:32.330366 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:32.330296 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:32.427388 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:32.427355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:32.527094 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:32.527004 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:32.626888 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:32.626857 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:33.118794 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:33.118757 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:50.238316 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:50.238274 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:05:59.041930 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:05:59.041893 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:06.151806 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:06.151776 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:06.344858 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:06.344812 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:06.449278 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:06.449198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:12.278009 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:12.277919 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:12.328800 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:12.328766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:22.396394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:22.396360 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:28.060438 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:28.060407 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:28.171352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:28.171324 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:28.260515 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:28.260477 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:38.852815 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:38.852762 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:39.908646 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:39.908610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:45.546137 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:45.546099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:45.653152 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:45.653121 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:45.770936 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:45.770894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:45.867771 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:45.867677 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:46.664000 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:46.663938 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:46.762087 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:46.762048 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:46.858606 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:46.858576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:48.786478 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:48.786435 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:52.293160 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:52.293123 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:06:54.628386 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:54.628347 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:54.728999 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:54.728958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:06:55.391443 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:06:55.391385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:00.905841 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:00.905800 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:07:01.516154 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:01.516108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:01.613269 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:01.613201 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:01.767022 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:01.766917 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:02.083471 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.083390 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc"] Jul 24 16:07:02.084018 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.083996 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" Jul 24 16:07:02.084108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084021 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" Jul 24 16:07:02.084108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084038 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" Jul 24 16:07:02.084108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084046 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" Jul 24 16:07:02.084108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084057 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afea36d4-f337-4d9c-8bd7-635b257603d8" containerName="revoke-keys" Jul 24 16:07:02.084108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084065 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="afea36d4-f337-4d9c-8bd7-635b257603d8" containerName="revoke-keys" Jul 24 16:07:02.084108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084077 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" Jul 24 16:07:02.084108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084084 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" Jul 24 16:07:02.084480 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084208 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="afea36d4-f337-4d9c-8bd7-635b257603d8" containerName="revoke-keys" Jul 24 16:07:02.084480 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084220 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3880b5cd-433b-4b9d-918b-6a1200a075ce" containerName="istio-proxy" Jul 24 16:07:02.084480 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084232 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" Jul 24 16:07:02.084480 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.084244 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13c0db05-f778-441a-a3f7-18acd7a245c1" containerName="istio-proxy" Jul 24 16:07:02.087370 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.087349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.090912 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.090895 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-850ff15f-openshift-default-dockercfg-wn2q9\"" Jul 24 16:07:02.112263 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.112234 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc"] Jul 24 16:07:02.162891 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.162854 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:02.166726 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.166698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-token\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.166867 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.166772 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwthr\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-kube-api-access-cwthr\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.166867 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.166805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a553c041-90e7-4a11-9c9a-179944f8b56b-istiod-ca-cert\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.166867 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.166857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-socket\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.167043 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.166889 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-credential-socket\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.167043 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.166953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-podinfo\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.167043 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.166978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-data\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.167043 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.167034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-certs\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.167189 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.167057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-envoy\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268281 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268250 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-socket\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268281 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-credential-socket\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-podinfo\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268360 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-data\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-certs\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-envoy\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268449 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-token\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwthr\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-kube-api-access-cwthr\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a553c041-90e7-4a11-9c9a-179944f8b56b-istiod-ca-cert\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268942 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268717 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-socket\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268942 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268744 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-credential-socket\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268942 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-data\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.268942 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.268880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-certs\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.269368 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.269167 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a553c041-90e7-4a11-9c9a-179944f8b56b-istiod-ca-cert\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.270877 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.270853 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-envoy\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.271097 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.271078 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-podinfo\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.284933 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.284899 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-token\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.285033 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.284976 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwthr\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-kube-api-access-cwthr\") pod \"e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.398767 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.398690 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:02.540290 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.540263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc"] Jul 24 16:07:02.542488 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:07:02.542458 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda553c041_90e7_4a11_9c9a_179944f8b56b.slice/crio-ef338131111830e7673d249f46ec61fe6c8f534dc3c69b131af6cb88bef47442 WatchSource:0}: Error finding container ef338131111830e7673d249f46ec61fe6c8f534dc3c69b131af6cb88bef47442: Status 404 returned error can't find the container with id ef338131111830e7673d249f46ec61fe6c8f534dc3c69b131af6cb88bef47442 Jul 24 16:07:02.544543 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.544510 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:02.544613 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.544576 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:02.544613 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:02.544601 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:03.098754 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:03.098720 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:03.218677 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:03.218641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" event={"ID":"a553c041-90e7-4a11-9c9a-179944f8b56b","Type":"ContainerStarted","Data":"e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4"} Jul 24 16:07:03.218844 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:03.218691 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" event={"ID":"a553c041-90e7-4a11-9c9a-179944f8b56b","Type":"ContainerStarted","Data":"ef338131111830e7673d249f46ec61fe6c8f534dc3c69b131af6cb88bef47442"} Jul 24 16:07:03.245976 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:03.245929 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" podStartSLOduration=1.245912192 podStartE2EDuration="1.245912192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:07:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:07:03.243067622 +0000 UTC m=+1846.665668666" watchObservedRunningTime="2026-07-24 16:07:03.245912192 +0000 UTC m=+1846.668513306" Jul 24 16:07:03.399794 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:03.399690 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:03.404676 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:03.404652 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:04.223843 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:04.223811 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:04.224855 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:04.224837 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:05.294134 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:05.294107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:05.389271 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:05.389241 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:05.480796 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:05.480763 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:05.587401 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:05.587317 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:06.054051 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:06.053998 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:06.217715 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:06.217675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:09.807291 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:09.807252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:10.530511 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:10.530480 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:10.632008 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:10.631968 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:10.965006 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:10.964830 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:16.092882 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:16.092849 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:07:16.885366 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:16.885331 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:16.988210 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:16.988179 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:17.276319 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.276262 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr"] Jul 24 16:07:17.279545 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.279528 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" Jul 24 16:07:17.282023 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.282005 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:07:17.282133 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.282050 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:07:17.288750 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.288717 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr"] Jul 24 16:07:17.317219 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.317180 2560 generic.go:356] "Generic (PLEG): container finished" podID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerID="e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4" exitCode=0 Jul 24 16:07:17.317415 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.317237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" event={"ID":"a553c041-90e7-4a11-9c9a-179944f8b56b","Type":"ContainerDied","Data":"e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4"} Jul 24 16:07:17.317820 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.317796 2560 scope.go:117] "RemoveContainer" containerID="e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4" Jul 24 16:07:17.319234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.319193 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:17.319296 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.319283 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:17.319381 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.319362 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:17.433417 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.433384 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a60b19-50d1-46d1-96c0-e32a2f09d5ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr\" (UID: \"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" Jul 24 16:07:17.534838 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.534740 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a60b19-50d1-46d1-96c0-e32a2f09d5ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr\" (UID: \"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" Jul 24 16:07:17.535378 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.535357 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a60b19-50d1-46d1-96c0-e32a2f09d5ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr\" (UID: \"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" Jul 24 16:07:17.590915 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.590885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" Jul 24 16:07:17.737957 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:17.737919 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr"] Jul 24 16:07:17.741069 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:07:17.741042 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a60b19_50d1_46d1_96c0_e32a2f09d5ea.slice/crio-81c0be6986e501b9ed25aa0b00fb174bbdb563290dbd911f93e1e479954d001a WatchSource:0}: Error finding container 81c0be6986e501b9ed25aa0b00fb174bbdb563290dbd911f93e1e479954d001a: Status 404 returned error can't find the container with id 81c0be6986e501b9ed25aa0b00fb174bbdb563290dbd911f93e1e479954d001a Jul 24 16:07:18.327447 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:18.327405 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" event={"ID":"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea","Type":"ContainerStarted","Data":"5b53cbf1693b46478150ac2d9859e835e241e9392d05de2d43bc64e52b7a0c94"} Jul 24 16:07:18.327900 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:18.327475 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" event={"ID":"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea","Type":"ContainerStarted","Data":"81c0be6986e501b9ed25aa0b00fb174bbdb563290dbd911f93e1e479954d001a"} Jul 24 16:07:18.329909 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:18.329876 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" event={"ID":"a553c041-90e7-4a11-9c9a-179944f8b56b","Type":"ContainerStarted","Data":"dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f"} Jul 24 16:07:18.343649 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:18.343562 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" podStartSLOduration=1.343541179 podStartE2EDuration="1.343541179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:07:18.343066857 +0000 UTC m=+1861.765667909" watchObservedRunningTime="2026-07-24 16:07:18.343541179 +0000 UTC m=+1861.766142210" Jul 24 16:07:18.398835 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:18.398793 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:18.400075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:18.400046 2560 patch_prober.go:28] interesting pod/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 16:07:18.400213 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:18.400110 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 16:07:19.224737 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:19.224703 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:19.335873 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:19.335839 2560 generic.go:356] "Generic (PLEG): container finished" podID="b7a60b19-50d1-46d1-96c0-e32a2f09d5ea" containerID="5b53cbf1693b46478150ac2d9859e835e241e9392d05de2d43bc64e52b7a0c94" exitCode=0 Jul 24 16:07:19.336255 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:19.335930 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" event={"ID":"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea","Type":"ContainerDied","Data":"5b53cbf1693b46478150ac2d9859e835e241e9392d05de2d43bc64e52b7a0c94"} Jul 24 16:07:19.399418 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:19.399377 2560 patch_prober.go:28] interesting pod/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 16:07:19.399622 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:19.399439 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 16:07:20.399901 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:20.399858 2560 patch_prober.go:28] interesting pod/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 16:07:20.400281 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:20.399936 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 16:07:20.509590 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:20.509561 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" Jul 24 16:07:20.680407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:20.679684 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a60b19-50d1-46d1-96c0-e32a2f09d5ea-maas-api-service-ca\") pod \"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea\" (UID: \"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea\") " Jul 24 16:07:20.680407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:20.680284 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7a60b19-50d1-46d1-96c0-e32a2f09d5ea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7a60b19-50d1-46d1-96c0-e32a2f09d5ea" (UID: "b7a60b19-50d1-46d1-96c0-e32a2f09d5ea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:07:20.780918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:20.780871 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a60b19-50d1-46d1-96c0-e32a2f09d5ea-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:21.347845 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:21.347812 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" Jul 24 16:07:21.348021 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:21.347842 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr" event={"ID":"b7a60b19-50d1-46d1-96c0-e32a2f09d5ea","Type":"ContainerDied","Data":"81c0be6986e501b9ed25aa0b00fb174bbdb563290dbd911f93e1e479954d001a"} Jul 24 16:07:21.348021 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:21.347889 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81c0be6986e501b9ed25aa0b00fb174bbdb563290dbd911f93e1e479954d001a" Jul 24 16:07:21.400070 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:21.399968 2560 patch_prober.go:28] interesting pod/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 16:07:21.400070 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:21.400035 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 16:07:22.400061 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:22.400021 2560 patch_prober.go:28] interesting pod/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 24 16:07:22.400282 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:22.400092 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 24 16:07:23.403594 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:23.403562 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:23.404755 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:23.404732 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:27.759715 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:27.759680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:27.859117 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:27.859081 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:27.960530 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:27.960490 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:31.107370 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:31.107321 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:07:32.365428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:32.365398 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:32.371889 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:32.371864 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr"] Jul 24 16:07:32.373584 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:32.373562 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-850ff15f-qjqlr"] Jul 24 16:07:32.838568 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:32.838508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:33.141071 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:33.140988 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7a60b19-50d1-46d1-96c0-e32a2f09d5ea" path="/var/lib/kubelet/pods/b7a60b19-50d1-46d1-96c0-e32a2f09d5ea/volumes" Jul 24 16:07:38.784756 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:38.784660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:38.846803 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:38.846766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:39.043604 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:39.043519 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc"] Jul 24 16:07:39.043820 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:39.043794 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" containerID="cri-o://dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f" gracePeriod=30 Jul 24 16:07:39.065967 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:39.065938 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:39.311472 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:39.311376 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:39.457879 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:39.457824 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:39.552618 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:39.552575 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:39.672217 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:39.672123 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:44.287916 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.287889 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:44.409547 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409459 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-token\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409547 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409506 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwthr\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-kube-api-access-cwthr\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409547 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409541 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-certs\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409597 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-data\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409615 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-envoy\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409633 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-credential-socket\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409687 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-socket\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409711 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-podinfo\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.409848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409738 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a553c041-90e7-4a11-9c9a-179944f8b56b-istiod-ca-cert\") pod \"a553c041-90e7-4a11-9c9a-179944f8b56b\" (UID: \"a553c041-90e7-4a11-9c9a-179944f8b56b\") " Jul 24 16:07:44.410156 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409893 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:07:44.410156 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409962 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:07:44.410156 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.409974 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:07:44.410521 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.410265 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a553c041-90e7-4a11-9c9a-179944f8b56b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:07:44.410592 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.410533 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.410592 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.410553 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.410592 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.410577 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a553c041-90e7-4a11-9c9a-179944f8b56b-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.410750 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.410589 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.410750 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.410720 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-data" (OuterVolumeSpecName: "istio-data") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:07:44.412000 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.411971 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-kube-api-access-cwthr" (OuterVolumeSpecName: "kube-api-access-cwthr") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "kube-api-access-cwthr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:07:44.412000 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.411990 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:07:44.412461 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.412438 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-token" (OuterVolumeSpecName: "istio-token") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:07:44.412533 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.412459 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a553c041-90e7-4a11-9c9a-179944f8b56b" (UID: "a553c041-90e7-4a11-9c9a-179944f8b56b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:07:44.487806 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.487775 2560 generic.go:356] "Generic (PLEG): container finished" podID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerID="dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f" exitCode=0 Jul 24 16:07:44.487993 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.487825 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" Jul 24 16:07:44.487993 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.487863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" event={"ID":"a553c041-90e7-4a11-9c9a-179944f8b56b","Type":"ContainerDied","Data":"dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f"} Jul 24 16:07:44.487993 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.487918 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc" event={"ID":"a553c041-90e7-4a11-9c9a-179944f8b56b","Type":"ContainerDied","Data":"ef338131111830e7673d249f46ec61fe6c8f534dc3c69b131af6cb88bef47442"} Jul 24 16:07:44.487993 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.487946 2560 scope.go:117] "RemoveContainer" containerID="dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f" Jul 24 16:07:44.497369 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.497351 2560 scope.go:117] "RemoveContainer" containerID="e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4" Jul 24 16:07:44.505657 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.505641 2560 scope.go:117] "RemoveContainer" containerID="dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f" Jul 24 16:07:44.505924 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:07:44.505904 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f\": container with ID starting with dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f not found: ID does not exist" containerID="dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f" Jul 24 16:07:44.505998 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.505931 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f"} err="failed to get container status \"dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f\": rpc error: code = NotFound desc = could not find container \"dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f\": container with ID starting with dfc6961af71632ed8b64036d088cf1296f0b1ae02bc80f68800bcebc7936125f not found: ID does not exist" Jul 24 16:07:44.505998 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.505948 2560 scope.go:117] "RemoveContainer" containerID="e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4" Jul 24 16:07:44.506199 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:07:44.506182 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4\": container with ID starting with e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4 not found: ID does not exist" containerID="e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4" Jul 24 16:07:44.506240 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.506207 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4"} err="failed to get container status \"e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4\": rpc error: code = NotFound desc = could not find container \"e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4\": container with ID starting with e8a826d313ba50acb51c4e282d808716bb6a73c4b7828eb2bfd4faf85bbc49c4 not found: ID does not exist" Jul 24 16:07:44.510245 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.510222 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc"] Jul 24 16:07:44.511580 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.511563 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.511643 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.511582 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.511643 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.511590 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.511643 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.511598 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.511643 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.511610 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwthr\" (UniqueName: \"kubernetes.io/projected/a553c041-90e7-4a11-9c9a-179944f8b56b-kube-api-access-cwthr\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:07:44.514425 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:44.514403 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-850ff15f-openshift-default-76cb4bbdb8-qw2kc"] Jul 24 16:07:45.140789 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:45.140756 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" path="/var/lib/kubelet/pods/a553c041-90e7-4a11-9c9a-179944f8b56b/volumes" Jul 24 16:07:45.583370 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:45.583338 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:45.693941 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:45.693912 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:45.784485 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:45.784445 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:46.034603 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:46.034569 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:52.134311 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:52.134279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:52.233973 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:52.233942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:59.059243 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.059205 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:59.362999 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.362922 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z"] Jul 24 16:07:59.363347 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363333 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" Jul 24 16:07:59.363399 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363348 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" Jul 24 16:07:59.363399 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363359 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" Jul 24 16:07:59.363399 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363365 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" Jul 24 16:07:59.363399 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363383 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a60b19-50d1-46d1-96c0-e32a2f09d5ea" containerName="revoke-keys" Jul 24 16:07:59.363399 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363388 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a60b19-50d1-46d1-96c0-e32a2f09d5ea" containerName="revoke-keys" Jul 24 16:07:59.363553 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363447 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" Jul 24 16:07:59.363553 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363456 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a60b19-50d1-46d1-96c0-e32a2f09d5ea" containerName="revoke-keys" Jul 24 16:07:59.363553 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.363462 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a553c041-90e7-4a11-9c9a-179944f8b56b" containerName="istio-proxy" Jul 24 16:07:59.366738 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.366720 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.369728 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.369704 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5b382fb6-openshift-default-dockercfg-8hrbp\"" Jul 24 16:07:59.378246 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.378222 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z"] Jul 24 16:07:59.401662 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.401626 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:07:59.559354 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istiod-ca-cert\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-data\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-socket\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559443 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-podinfo\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-token\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-certs\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qngbb\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-kube-api-access-qngbb\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559788 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559567 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-credential-socket\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.559788 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.559582 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-envoy\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.660821 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.660719 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-socket\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.660821 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.660761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-podinfo\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.660821 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.660791 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-token\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661107 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.660827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-certs\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661107 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.660892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qngbb\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-kube-api-access-qngbb\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661107 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.660926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-credential-socket\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661107 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.660949 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-envoy\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.661113 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istiod-ca-cert\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.661152 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-socket\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.661169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-data\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.661262 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-certs\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.661341 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-credential-socket\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.661590 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.661493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-data\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.662348 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.662326 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istiod-ca-cert\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.663274 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.663253 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-envoy\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.663575 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.663560 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-podinfo\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.670585 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.670555 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-token\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.670729 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.670706 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qngbb\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-kube-api-access-qngbb\") pod \"e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.679144 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.679119 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:07:59.821223 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.821197 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z"] Jul 24 16:07:59.823643 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:07:59.823608 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae31d8f_5567_41c0_bbf1_7dc89aedb00f.slice/crio-4a2115d3c565d68d45d3207c43b96ef54478cf70bf2e0c5b1d07ee15f8668b88 WatchSource:0}: Error finding container 4a2115d3c565d68d45d3207c43b96ef54478cf70bf2e0c5b1d07ee15f8668b88: Status 404 returned error can't find the container with id 4a2115d3c565d68d45d3207c43b96ef54478cf70bf2e0c5b1d07ee15f8668b88 Jul 24 16:07:59.826266 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.826219 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:59.826386 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.826351 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:07:59.826435 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:07:59.826404 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:08:00.016595 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:00.016554 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:00.574464 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:00.574429 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" event={"ID":"cae31d8f-5567-41c0-bbf1-7dc89aedb00f","Type":"ContainerStarted","Data":"a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594"} Jul 24 16:08:00.574887 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:00.574479 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" event={"ID":"cae31d8f-5567-41c0-bbf1-7dc89aedb00f","Type":"ContainerStarted","Data":"4a2115d3c565d68d45d3207c43b96ef54478cf70bf2e0c5b1d07ee15f8668b88"} Jul 24 16:08:00.596099 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:00.596038 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" podStartSLOduration=1.5960181439999999 podStartE2EDuration="1.596018144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:08:00.594091732 +0000 UTC m=+1904.016692786" watchObservedRunningTime="2026-07-24 16:08:00.596018144 +0000 UTC m=+1904.018619174" Jul 24 16:08:00.680256 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:00.680228 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:00.684974 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:00.684953 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:01.579801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:01.579769 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:01.580831 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:01.580808 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:02.584690 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:02.584654 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:02.828558 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:02.828521 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:02.945281 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:02.945151 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:03.184585 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:03.184554 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:03.438498 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:03.438458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:03.538461 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:03.538427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:07.103412 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:07.103377 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:10.006803 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:10.006760 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:08:12.790419 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:12.790136 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:17.621923 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:17.621839 2560 patch_prober.go:28] interesting pod/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 16:08:17.622444 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:17.622111 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 16:08:18.002439 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:18.002397 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:08:18.671176 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:18.671142 2560 generic.go:356] "Generic (PLEG): container finished" podID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerID="a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594" exitCode=0 Jul 24 16:08:18.671679 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:18.671229 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" event={"ID":"cae31d8f-5567-41c0-bbf1-7dc89aedb00f","Type":"ContainerDied","Data":"a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594"} Jul 24 16:08:18.671679 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:18.671657 2560 scope.go:117] "RemoveContainer" containerID="a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594" Jul 24 16:08:18.672739 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:18.672707 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:08:18.672808 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:18.672781 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:08:18.672852 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:18.672816 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:08:19.682728 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:19.681952 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" event={"ID":"cae31d8f-5567-41c0-bbf1-7dc89aedb00f","Type":"ContainerStarted","Data":"078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6"} Jul 24 16:08:20.680027 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:20.679984 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:20.680494 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:20.680447 2560 patch_prober.go:28] interesting pod/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 24 16:08:20.680647 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:20.680509 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 24 16:08:21.680623 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:21.680063 2560 patch_prober.go:28] interesting pod/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 24 16:08:21.680623 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:21.680141 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 24 16:08:22.680490 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:22.680445 2560 patch_prober.go:28] interesting pod/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 24 16:08:22.680717 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:22.680521 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 24 16:08:23.683771 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:23.683741 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:23.684191 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:23.684014 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:23.684589 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:23.684571 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:08:31.998278 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:31.998233 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:08:36.630384 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.630343 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn"] Jul 24 16:08:36.633127 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.633105 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" Jul 24 16:08:36.635826 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.635802 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:08:36.635826 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.635820 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:08:36.642953 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.642925 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn"] Jul 24 16:08:36.732189 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.732139 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f7c744-7ba5-48cf-8d88-6579c6fa8277-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn\" (UID: \"f4f7c744-7ba5-48cf-8d88-6579c6fa8277\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" Jul 24 16:08:36.833553 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.833512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f7c744-7ba5-48cf-8d88-6579c6fa8277-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn\" (UID: \"f4f7c744-7ba5-48cf-8d88-6579c6fa8277\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" Jul 24 16:08:36.834138 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.834119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f7c744-7ba5-48cf-8d88-6579c6fa8277-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn\" (UID: \"f4f7c744-7ba5-48cf-8d88-6579c6fa8277\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" Jul 24 16:08:36.944336 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:36.944225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" Jul 24 16:08:37.084585 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:37.078649 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn"] Jul 24 16:08:37.770689 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:37.770649 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" event={"ID":"f4f7c744-7ba5-48cf-8d88-6579c6fa8277","Type":"ContainerStarted","Data":"78f12fd026729ff62ca9859b611159c95ca5abac3f201f9bfa5a19f9830e94a9"} Jul 24 16:08:37.771073 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:37.770709 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" event={"ID":"f4f7c744-7ba5-48cf-8d88-6579c6fa8277","Type":"ContainerStarted","Data":"7494fd746204050bd6e3b7369cc577f3269724b9d03a8f7d927a3c13592092be"} Jul 24 16:08:37.786397 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:37.786347 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" podStartSLOduration=1.786330432 podStartE2EDuration="1.786330432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:08:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:08:37.785168589 +0000 UTC m=+1941.207769642" watchObservedRunningTime="2026-07-24 16:08:37.786330432 +0000 UTC m=+1941.208931461" Jul 24 16:08:38.777479 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:38.777443 2560 generic.go:356] "Generic (PLEG): container finished" podID="f4f7c744-7ba5-48cf-8d88-6579c6fa8277" containerID="78f12fd026729ff62ca9859b611159c95ca5abac3f201f9bfa5a19f9830e94a9" exitCode=0 Jul 24 16:08:38.777856 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:38.777527 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" event={"ID":"f4f7c744-7ba5-48cf-8d88-6579c6fa8277","Type":"ContainerDied","Data":"78f12fd026729ff62ca9859b611159c95ca5abac3f201f9bfa5a19f9830e94a9"} Jul 24 16:08:39.917436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:39.917411 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" Jul 24 16:08:39.964174 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:39.964145 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f7c744-7ba5-48cf-8d88-6579c6fa8277-maas-api-service-ca\") pod \"f4f7c744-7ba5-48cf-8d88-6579c6fa8277\" (UID: \"f4f7c744-7ba5-48cf-8d88-6579c6fa8277\") " Jul 24 16:08:39.964514 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:39.964493 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f7c744-7ba5-48cf-8d88-6579c6fa8277-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4f7c744-7ba5-48cf-8d88-6579c6fa8277" (UID: "f4f7c744-7ba5-48cf-8d88-6579c6fa8277"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:08:40.064936 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:40.064843 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f7c744-7ba5-48cf-8d88-6579c6fa8277-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:08:40.788463 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:40.788433 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" Jul 24 16:08:40.788663 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:40.788474 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn" event={"ID":"f4f7c744-7ba5-48cf-8d88-6579c6fa8277","Type":"ContainerDied","Data":"7494fd746204050bd6e3b7369cc577f3269724b9d03a8f7d927a3c13592092be"} Jul 24 16:08:40.788663 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:40.788518 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7494fd746204050bd6e3b7369cc577f3269724b9d03a8f7d927a3c13592092be" Jul 24 16:08:50.590855 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:50.590819 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:08:52.112106 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:52.112075 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:52.178339 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:52.178283 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:52.279471 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:52.279439 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:56.703063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:56.703019 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn"] Jul 24 16:08:56.704703 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:56.704673 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b382fb6-vt8wn"] Jul 24 16:08:56.705453 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:56.705430 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:08:57.141555 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:08:57.141522 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f7c744-7ba5-48cf-8d88-6579c6fa8277" path="/var/lib/kubelet/pods/f4f7c744-7ba5-48cf-8d88-6579c6fa8277/volumes" Jul 24 16:09:00.159889 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:00.159852 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:09:01.793956 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:01.793917 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:09:05.195835 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:05.195796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:09:14.257186 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:14.257150 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:09:17.219085 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:17.219049 2560 scope.go:117] "RemoveContainer" containerID="af842c96e41b5f1eb976ccbff27b98aea1c0f0a4eb2253a79123f11db2e0b11f" Jul 24 16:09:42.075013 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:42.074977 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:09:42.518700 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:42.518663 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:09:48.023351 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:09:48.023296 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:02.874719 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:02.874681 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:02.974332 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:02.974266 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:03.141672 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:03.141597 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z"] Jul 24 16:10:03.141822 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:03.141796 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" containerID="cri-o://078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6" gracePeriod=30 Jul 24 16:10:03.172324 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:03.172271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:03.431500 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:03.431409 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:04.617448 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:04.617416 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:04.712407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:04.712372 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:04.810422 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:04.810381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:05.659023 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:05.658986 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:05.758407 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:05.758373 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:05.858355 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:05.858292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:07.378928 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:07.378898 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:07.504579 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:07.504542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:10:08.404117 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.404092 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:10:08.547007 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.546968 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-certs\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.547214 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547020 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qngbb\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-kube-api-access-qngbb\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.547214 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547064 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-socket\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.547214 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547109 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-podinfo\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.547214 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547171 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istiod-ca-cert\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.547542 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547324 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:10:08.547542 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547478 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:10:08.547738 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547714 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:10:08.547738 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547728 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-token\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.547897 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547806 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-envoy\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.547897 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547848 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-data\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.548012 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.547917 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-credential-socket\") pod \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\" (UID: \"cae31d8f-5567-41c0-bbf1-7dc89aedb00f\") " Jul 24 16:10:08.549492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.548264 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.549492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.548290 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.549492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.548337 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.549492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.549395 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-data" (OuterVolumeSpecName: "istio-data") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:10:08.549492 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.549465 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:10:08.550103 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.550071 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:10:08.550382 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.550355 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-kube-api-access-qngbb" (OuterVolumeSpecName: "kube-api-access-qngbb") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "kube-api-access-qngbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:10:08.551087 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.551051 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-token" (OuterVolumeSpecName: "istio-token") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:10:08.551850 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.551825 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cae31d8f-5567-41c0-bbf1-7dc89aedb00f" (UID: "cae31d8f-5567-41c0-bbf1-7dc89aedb00f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:10:08.649595 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.649560 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.649595 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.649589 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.649595 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.649597 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.649595 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.649605 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.649859 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.649615 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qngbb\" (UniqueName: \"kubernetes.io/projected/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-kube-api-access-qngbb\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:08.649859 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:08.649623 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cae31d8f-5567-41c0-bbf1-7dc89aedb00f-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:10:09.239850 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.239813 2560 generic.go:356] "Generic (PLEG): container finished" podID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerID="078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6" exitCode=0 Jul 24 16:10:09.240045 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.239861 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" event={"ID":"cae31d8f-5567-41c0-bbf1-7dc89aedb00f","Type":"ContainerDied","Data":"078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6"} Jul 24 16:10:09.240045 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.239872 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" Jul 24 16:10:09.240045 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.239930 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z" event={"ID":"cae31d8f-5567-41c0-bbf1-7dc89aedb00f","Type":"ContainerDied","Data":"4a2115d3c565d68d45d3207c43b96ef54478cf70bf2e0c5b1d07ee15f8668b88"} Jul 24 16:10:09.240045 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.239959 2560 scope.go:117] "RemoveContainer" containerID="078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6" Jul 24 16:10:09.249050 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.249032 2560 scope.go:117] "RemoveContainer" containerID="a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594" Jul 24 16:10:09.257288 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.257267 2560 scope.go:117] "RemoveContainer" containerID="078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6" Jul 24 16:10:09.257613 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:10:09.257588 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6\": container with ID starting with 078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6 not found: ID does not exist" containerID="078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6" Jul 24 16:10:09.257673 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.257626 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6"} err="failed to get container status \"078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6\": rpc error: code = NotFound desc = could not find container \"078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6\": container with ID starting with 078278298ec878426986a214f4019a7beba8ae544bbcb262c025b87a2c98bcc6 not found: ID does not exist" Jul 24 16:10:09.257673 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.257645 2560 scope.go:117] "RemoveContainer" containerID="a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594" Jul 24 16:10:09.257889 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:10:09.257864 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594\": container with ID starting with a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594 not found: ID does not exist" containerID="a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594" Jul 24 16:10:09.257983 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.257893 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594"} err="failed to get container status \"a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594\": rpc error: code = NotFound desc = could not find container \"a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594\": container with ID starting with a1e65742cb1deb27b1ed9584f287f132f6d991210ce7bf137cbcea5f5bfa1594 not found: ID does not exist" Jul 24 16:10:09.261974 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.261949 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z"] Jul 24 16:10:09.265695 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:09.265671 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5b382fb6-openshift-default-599787cdbd-g8l2z"] Jul 24 16:10:11.141572 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:11.141541 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" path="/var/lib/kubelet/pods/cae31d8f-5567-41c0-bbf1-7dc89aedb00f/volumes" Jul 24 16:10:13.327132 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:13.327096 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:13.427422 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:13.427390 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:14.052359 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:14.052325 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:17.241548 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:17.241514 2560 scope.go:117] "RemoveContainer" containerID="c8f69ad5439ae4d717866128c6312c66d8657bfee55e0bc487318cd1a2f0c52a" Jul 24 16:10:19.990238 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:19.990201 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:20.087718 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:20.087671 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:20.184583 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:20.184546 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:20.499375 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:20.499342 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:10:30.791395 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:30.791324 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:31.174913 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:31.174822 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:52.481131 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:52.481049 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:58.560925 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:58.560890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:58.659984 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:58.659950 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:58.999522 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:58.999437 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:59.298369 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298337 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp"] Jul 24 16:10:59.298739 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298726 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" Jul 24 16:10:59.298796 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298741 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" Jul 24 16:10:59.298796 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298757 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" Jul 24 16:10:59.298796 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298762 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" Jul 24 16:10:59.298796 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298768 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f7c744-7ba5-48cf-8d88-6579c6fa8277" containerName="revoke-keys" Jul 24 16:10:59.298796 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298787 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f7c744-7ba5-48cf-8d88-6579c6fa8277" containerName="revoke-keys" Jul 24 16:10:59.298967 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298862 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" Jul 24 16:10:59.298967 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298871 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cae31d8f-5567-41c0-bbf1-7dc89aedb00f" containerName="istio-proxy" Jul 24 16:10:59.298967 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.298879 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f7c744-7ba5-48cf-8d88-6579c6fa8277" containerName="revoke-keys" Jul 24 16:10:59.302253 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.302227 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.304820 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.304797 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e03b82-openshift-default-dockercfg-9t4gn\"" Jul 24 16:10:59.325608 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325574 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-data\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.325775 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-podinfo\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.325775 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-token\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.325775 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-envoy\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.325948 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325827 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istiod-ca-cert\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.325948 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325893 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-socket\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.325948 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-certs\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.325948 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325942 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtln\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-kube-api-access-jmtln\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.326147 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.325966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-credential-socket\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.331908 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.331884 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:59.371692 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.371658 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp"] Jul 24 16:10:59.426518 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-socket\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.426518 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-certs\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.426805 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtln\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-kube-api-access-jmtln\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.426805 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-credential-socket\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.426805 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-data\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.426805 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-podinfo\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426824 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-token\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-envoy\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426928 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-socket\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.426961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istiod-ca-cert\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.427047 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-credential-socket\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427381 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.427237 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-data\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427381 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.427323 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-certs\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.427654 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.427632 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istiod-ca-cert\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.429067 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.429044 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-envoy\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.429186 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.429145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-podinfo\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.434869 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.434836 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-token\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.434963 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.434916 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtln\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-kube-api-access-jmtln\") pod \"e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.590590 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.590471 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:10:59.612550 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.612518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:10:59.760635 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.760607 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp"] Jul 24 16:10:59.762287 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:10:59.762256 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e4b035_3b9a_4bd6_b199_85ea60214ca6.slice/crio-3005a0b5fbba09d40eef61c19184b11385fc064dbbdabbbd52f7bd07a85f5a39 WatchSource:0}: Error finding container 3005a0b5fbba09d40eef61c19184b11385fc064dbbdabbbd52f7bd07a85f5a39: Status 404 returned error can't find the container with id 3005a0b5fbba09d40eef61c19184b11385fc064dbbdabbbd52f7bd07a85f5a39 Jul 24 16:10:59.764555 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.764513 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:10:59.764692 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.764578 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:10:59.764692 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:10:59.764606 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:00.491810 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:00.491768 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" event={"ID":"d9e4b035-3b9a-4bd6-b199-85ea60214ca6","Type":"ContainerStarted","Data":"93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718"} Jul 24 16:11:00.492002 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:00.491822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" event={"ID":"d9e4b035-3b9a-4bd6-b199-85ea60214ca6","Type":"ContainerStarted","Data":"3005a0b5fbba09d40eef61c19184b11385fc064dbbdabbbd52f7bd07a85f5a39"} Jul 24 16:11:00.516709 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:00.516656 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" podStartSLOduration=1.516640326 podStartE2EDuration="1.516640326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:00.515691563 +0000 UTC m=+2083.938292609" watchObservedRunningTime="2026-07-24 16:11:00.516640326 +0000 UTC m=+2083.939241374" Jul 24 16:11:00.613497 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:00.613458 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:00.618164 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:00.618137 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:01.496548 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:01.496514 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:01.497355 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:01.497335 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:02.482033 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:02.481966 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:02.583387 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:02.583350 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:02.699738 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:02.699706 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:02.869183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:02.869150 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:03.180319 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:03.180212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:03.274212 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:03.274182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:07.039244 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.039200 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:07.436005 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.435910 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:07.726781 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.726688 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w"] Jul 24 16:11:07.730860 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.730835 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.733603 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.733577 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-aedd03-openshift-default-dockercfg-qjbvs\"" Jul 24 16:11:07.758283 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.758247 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w"] Jul 24 16:11:07.771003 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.770973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:07.807641 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-envoy\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807641 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a106356-f753-4f80-807a-8df96cff5eb3-istio-podinfo\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807872 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a106356-f753-4f80-807a-8df96cff5eb3-istiod-ca-cert\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807872 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-istio-token\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807872 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-credential-socket\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807872 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-socket\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807998 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-kube-api-access-8cxjl\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807998 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-certs\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.807998 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.807966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-data\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.877814 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.877780 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:07.910177 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.910137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-socket\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.910491 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.910467 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-kube-api-access-8cxjl\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.910719 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.910702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-certs\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.910840 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.910824 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-data\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.910969 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.910955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-envoy\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.911087 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.911073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a106356-f753-4f80-807a-8df96cff5eb3-istio-podinfo\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.911216 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.911201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a106356-f753-4f80-807a-8df96cff5eb3-istiod-ca-cert\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.911354 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.911339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-istio-token\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.911487 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.911471 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-credential-socket\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.911653 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.910737 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-socket\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.911768 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.910971 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-certs\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.911903 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.911213 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-data\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.913042 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.911984 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-credential-socket\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.913178 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.913104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a106356-f753-4f80-807a-8df96cff5eb3-istiod-ca-cert\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.914409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.914372 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-envoy\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.915444 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.915423 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a106356-f753-4f80-807a-8df96cff5eb3-istio-podinfo\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.926539 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.926479 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-istio-token\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:07.926682 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:07.926599 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-kube-api-access-8cxjl\") pod \"e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:08.042473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.042438 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:08.184848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.184816 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w"] Jul 24 16:11:08.188321 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:11:08.188276 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a106356_f753_4f80_807a_8df96cff5eb3.slice/crio-c5223d45f0316b6ff8cc5fe43b00aa676ea6b20e345c6e5ad4d6df0d46fac392 WatchSource:0}: Error finding container c5223d45f0316b6ff8cc5fe43b00aa676ea6b20e345c6e5ad4d6df0d46fac392: Status 404 returned error can't find the container with id c5223d45f0316b6ff8cc5fe43b00aa676ea6b20e345c6e5ad4d6df0d46fac392 Jul 24 16:11:08.190461 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.190428 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:08.190568 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.190507 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:08.190568 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.190549 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:08.531727 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.531685 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" event={"ID":"4a106356-f753-4f80-807a-8df96cff5eb3","Type":"ContainerStarted","Data":"f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f"} Jul 24 16:11:08.531925 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.531766 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" event={"ID":"4a106356-f753-4f80-807a-8df96cff5eb3","Type":"ContainerStarted","Data":"c5223d45f0316b6ff8cc5fe43b00aa676ea6b20e345c6e5ad4d6df0d46fac392"} Jul 24 16:11:08.558110 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:08.558066 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" podStartSLOduration=1.558049317 podStartE2EDuration="1.558049317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:08.556568825 +0000 UTC m=+2091.979169875" watchObservedRunningTime="2026-07-24 16:11:08.558049317 +0000 UTC m=+2091.980650351" Jul 24 16:11:09.042935 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:09.042902 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:09.047477 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:09.047453 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:09.536670 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:09.536646 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:09.537533 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:09.537515 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:10.565777 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:10.565720 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:10.664872 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:10.664838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:10.819807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:10.819707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:11.012280 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:11.012237 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:11.411814 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:11.411780 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:11.507968 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:11.507934 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:11.616786 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:11.616744 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:11:15.096524 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:15.096481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:17.265629 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:17.265598 2560 scope.go:117] "RemoveContainer" containerID="9f555f047b38b2252aa74cc4717ad8d73d6b7273ab9ce018223658e9d229478d" Jul 24 16:11:17.274020 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:17.274002 2560 scope.go:117] "RemoveContainer" containerID="a1a51d495ab2b87e6dc6a0a111949a7e91a0c78f9b9f9d6ebe249240532c97d0" Jul 24 16:11:20.599409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:20.599372 2560 generic.go:356] "Generic (PLEG): container finished" podID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerID="93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718" exitCode=0 Jul 24 16:11:20.599839 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:20.599434 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" event={"ID":"d9e4b035-3b9a-4bd6-b199-85ea60214ca6","Type":"ContainerDied","Data":"93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718"} Jul 24 16:11:20.599893 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:20.599878 2560 scope.go:117] "RemoveContainer" containerID="93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718" Jul 24 16:11:20.600914 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:20.600883 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:20.600990 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:20.600977 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:20.601026 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:20.601011 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:21.609911 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:21.609753 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" event={"ID":"d9e4b035-3b9a-4bd6-b199-85ea60214ca6","Type":"ContainerStarted","Data":"7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3"} Jul 24 16:11:21.612882 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:21.612859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:21.613072 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:21.613047 2560 patch_prober.go:28] interesting pod/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 24 16:11:21.613182 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:21.613100 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 24 16:11:22.613411 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:22.613366 2560 patch_prober.go:28] interesting pod/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 24 16:11:22.613848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:22.613448 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 24 16:11:22.840685 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:22.840645 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:23.612931 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:23.612879 2560 patch_prober.go:28] interesting pod/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 24 16:11:23.613194 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:23.612947 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 24 16:11:24.012666 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:24.012628 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:11:24.613146 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:24.613044 2560 patch_prober.go:28] interesting pod/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 24 16:11:24.613146 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:24.613117 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 24 16:11:25.578505 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:25.578460 2560 patch_prober.go:28] interesting pod/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 16:11:25.579019 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:25.578519 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 16:11:25.613144 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:25.613106 2560 patch_prober.go:28] interesting pod/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 24 16:11:25.613369 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:25.613163 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 24 16:11:26.617267 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:26.617238 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:26.617748 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:26.617641 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:26.618396 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:26.618379 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:11:27.642313 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:27.642257 2560 generic.go:356] "Generic (PLEG): container finished" podID="4a106356-f753-4f80-807a-8df96cff5eb3" containerID="f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f" exitCode=0 Jul 24 16:11:27.642754 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:27.642337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" event={"ID":"4a106356-f753-4f80-807a-8df96cff5eb3","Type":"ContainerDied","Data":"f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f"} Jul 24 16:11:27.642842 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:27.642826 2560 scope.go:117] "RemoveContainer" containerID="f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f" Jul 24 16:11:27.643902 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:27.643877 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:27.643997 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:27.643938 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:27.643997 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:27.643977 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:11:28.650784 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:28.650742 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" event={"ID":"4a106356-f753-4f80-807a-8df96cff5eb3","Type":"ContainerStarted","Data":"d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f"} Jul 24 16:11:29.043454 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:29.043416 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:29.043695 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:29.043670 2560 patch_prober.go:28] interesting pod/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 16:11:29.043756 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:29.043735 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 16:11:30.043357 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:30.043281 2560 patch_prober.go:28] interesting pod/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 16:11:30.043753 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:30.043383 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 16:11:31.043759 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:31.043716 2560 patch_prober.go:28] interesting pod/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 16:11:31.044175 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:31.043785 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 16:11:32.043778 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:32.043746 2560 patch_prober.go:28] interesting pod/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 16:11:32.044161 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:32.043801 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 16:11:33.047468 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:33.047436 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:33.047883 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:33.047811 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:33.048420 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:33.048402 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:11:39.496356 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:39.496313 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:11:41.827510 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:41.827473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:41.928040 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:41.928003 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:51.520409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:51.520361 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:51.599377 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:51.599333 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:53.090412 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:53.090377 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:11:55.442054 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:11:55.442013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:12:01.408921 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:01.408878 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:12:40.250198 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:40.250161 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:12:41.330319 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.330279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:12:41.430544 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.430499 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf"] Jul 24 16:12:41.434293 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.434269 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" Jul 24 16:12:41.437002 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.436984 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:12:41.437126 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.437011 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:12:41.442222 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.442191 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf"] Jul 24 16:12:41.582522 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.582428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01822220-cdf8-47b0-bdaa-0f6c3718363d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf\" (UID: \"01822220-cdf8-47b0-bdaa-0f6c3718363d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" Jul 24 16:12:41.683352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.683288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01822220-cdf8-47b0-bdaa-0f6c3718363d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf\" (UID: \"01822220-cdf8-47b0-bdaa-0f6c3718363d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" Jul 24 16:12:41.683937 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.683899 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01822220-cdf8-47b0-bdaa-0f6c3718363d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf\" (UID: \"01822220-cdf8-47b0-bdaa-0f6c3718363d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" Jul 24 16:12:41.745597 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.745559 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" Jul 24 16:12:41.884233 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:41.884200 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf"] Jul 24 16:12:41.886481 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:12:41.886453 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01822220_cdf8_47b0_bdaa_0f6c3718363d.slice/crio-647cee10b57dc7608c8ed62fcd80bc8dca377b501497c56d868247869a7c38f7 WatchSource:0}: Error finding container 647cee10b57dc7608c8ed62fcd80bc8dca377b501497c56d868247869a7c38f7: Status 404 returned error can't find the container with id 647cee10b57dc7608c8ed62fcd80bc8dca377b501497c56d868247869a7c38f7 Jul 24 16:12:42.029106 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:42.029068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" event={"ID":"01822220-cdf8-47b0-bdaa-0f6c3718363d","Type":"ContainerStarted","Data":"647cee10b57dc7608c8ed62fcd80bc8dca377b501497c56d868247869a7c38f7"} Jul 24 16:12:43.035697 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:43.035662 2560 generic.go:356] "Generic (PLEG): container finished" podID="01822220-cdf8-47b0-bdaa-0f6c3718363d" containerID="741cc1b3d80c85df33372c2933c1f99426381e7702ab788599cae451a547c5d0" exitCode=0 Jul 24 16:12:43.036130 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:43.035757 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" event={"ID":"01822220-cdf8-47b0-bdaa-0f6c3718363d","Type":"ContainerDied","Data":"741cc1b3d80c85df33372c2933c1f99426381e7702ab788599cae451a547c5d0"} Jul 24 16:12:44.175848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:44.175824 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" Jul 24 16:12:44.306782 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:44.306690 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01822220-cdf8-47b0-bdaa-0f6c3718363d-maas-api-service-ca\") pod \"01822220-cdf8-47b0-bdaa-0f6c3718363d\" (UID: \"01822220-cdf8-47b0-bdaa-0f6c3718363d\") " Jul 24 16:12:44.307069 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:44.307045 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01822220-cdf8-47b0-bdaa-0f6c3718363d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01822220-cdf8-47b0-bdaa-0f6c3718363d" (UID: "01822220-cdf8-47b0-bdaa-0f6c3718363d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:12:44.407914 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:44.407879 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01822220-cdf8-47b0-bdaa-0f6c3718363d-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:12:45.047730 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:45.047698 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" Jul 24 16:12:45.047920 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:45.047699 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf" event={"ID":"01822220-cdf8-47b0-bdaa-0f6c3718363d","Type":"ContainerDied","Data":"647cee10b57dc7608c8ed62fcd80bc8dca377b501497c56d868247869a7c38f7"} Jul 24 16:12:45.047920 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:45.047822 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="647cee10b57dc7608c8ed62fcd80bc8dca377b501497c56d868247869a7c38f7" Jul 24 16:12:45.500439 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:45.500349 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:12:52.128840 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:52.128801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:12:52.227377 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:52.227347 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:12:56.530988 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:56.530942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:12:56.539332 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:56.539282 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf"] Jul 24 16:12:56.543833 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:56.543810 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e03b82-mstbf"] Jul 24 16:12:56.961238 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:56.961158 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:12:57.140401 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:12:57.140367 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01822220-cdf8-47b0-bdaa-0f6c3718363d" path="/var/lib/kubelet/pods/01822220-cdf8-47b0-bdaa-0f6c3718363d/volumes" Jul 24 16:13:01.202788 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:01.202745 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:13:03.249545 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.249511 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:03.329329 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.329272 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:03.529021 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.528988 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp"] Jul 24 16:13:03.529221 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.529199 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" containerID="cri-o://7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3" gracePeriod=30 Jul 24 16:13:03.564800 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.564766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:03.801808 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.801721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:03.918497 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.918460 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd"] Jul 24 16:13:03.918989 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.918969 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01822220-cdf8-47b0-bdaa-0f6c3718363d" containerName="revoke-keys" Jul 24 16:13:03.919092 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.918991 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="01822220-cdf8-47b0-bdaa-0f6c3718363d" containerName="revoke-keys" Jul 24 16:13:03.919147 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.919125 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="01822220-cdf8-47b0-bdaa-0f6c3718363d" containerName="revoke-keys" Jul 24 16:13:03.922415 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.922394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" Jul 24 16:13:03.924953 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.924924 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:13:03.925068 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.924954 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:13:03.970120 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:03.970087 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd"] Jul 24 16:13:04.004161 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:04.004123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c9cac2-a9a4-4854-94ea-44d1033dc381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd\" (UID: \"e4c9cac2-a9a4-4854-94ea-44d1033dc381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" Jul 24 16:13:04.104889 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:04.104794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c9cac2-a9a4-4854-94ea-44d1033dc381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd\" (UID: \"e4c9cac2-a9a4-4854-94ea-44d1033dc381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" Jul 24 16:13:04.105516 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:04.105496 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c9cac2-a9a4-4854-94ea-44d1033dc381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd\" (UID: \"e4c9cac2-a9a4-4854-94ea-44d1033dc381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" Jul 24 16:13:04.232870 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:04.232832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" Jul 24 16:13:04.373583 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:04.373557 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd"] Jul 24 16:13:04.375940 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:13:04.375910 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c9cac2_a9a4_4854_94ea_44d1033dc381.slice/crio-088c60b1275d6caee1b9ec8583cf6df841c3467997541dd14efd8f4223c72fe4 WatchSource:0}: Error finding container 088c60b1275d6caee1b9ec8583cf6df841c3467997541dd14efd8f4223c72fe4: Status 404 returned error can't find the container with id 088c60b1275d6caee1b9ec8583cf6df841c3467997541dd14efd8f4223c72fe4 Jul 24 16:13:05.154359 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:05.154287 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4c9cac2-a9a4-4854-94ea-44d1033dc381" containerID="f44e0e6650218b34c42a37f383d60f3143c41c3836abea54559aa080ebd56b11" exitCode=0 Jul 24 16:13:05.154541 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:05.154382 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" event={"ID":"e4c9cac2-a9a4-4854-94ea-44d1033dc381","Type":"ContainerDied","Data":"f44e0e6650218b34c42a37f383d60f3143c41c3836abea54559aa080ebd56b11"} Jul 24 16:13:05.154541 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:05.154437 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" event={"ID":"e4c9cac2-a9a4-4854-94ea-44d1033dc381","Type":"ContainerStarted","Data":"088c60b1275d6caee1b9ec8583cf6df841c3467997541dd14efd8f4223c72fe4"} Jul 24 16:13:06.297465 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:06.297438 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" Jul 24 16:13:06.429482 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:06.429383 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c9cac2-a9a4-4854-94ea-44d1033dc381-maas-api-service-ca\") pod \"e4c9cac2-a9a4-4854-94ea-44d1033dc381\" (UID: \"e4c9cac2-a9a4-4854-94ea-44d1033dc381\") " Jul 24 16:13:06.429742 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:06.429719 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c9cac2-a9a4-4854-94ea-44d1033dc381-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4c9cac2-a9a4-4854-94ea-44d1033dc381" (UID: "e4c9cac2-a9a4-4854-94ea-44d1033dc381"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:13:06.429877 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:06.429857 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c9cac2-a9a4-4854-94ea-44d1033dc381-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:07.166778 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:07.166751 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" Jul 24 16:13:07.166939 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:07.166800 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd" event={"ID":"e4c9cac2-a9a4-4854-94ea-44d1033dc381","Type":"ContainerDied","Data":"088c60b1275d6caee1b9ec8583cf6df841c3467997541dd14efd8f4223c72fe4"} Jul 24 16:13:07.166939 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:07.166853 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="088c60b1275d6caee1b9ec8583cf6df841c3467997541dd14efd8f4223c72fe4" Jul 24 16:13:08.778626 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.778590 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:13:08.956659 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956563 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-data\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.956659 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956625 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istiod-ca-cert\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.956908 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956675 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmtln\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-kube-api-access-jmtln\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.956908 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956704 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-credential-socket\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.956908 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956874 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-certs\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.957075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956914 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-token\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.957075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956949 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-podinfo\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.957075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.956973 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-socket\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.957075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957000 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:08.957075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957017 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-envoy\") pod \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\" (UID: \"d9e4b035-3b9a-4bd6-b199-85ea60214ca6\") " Jul 24 16:13:08.957338 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957085 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:08.957338 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957101 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:13:08.957519 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957366 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:08.957519 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957491 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:08.957718 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957655 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:08.957718 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957678 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:08.957718 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957693 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:08.957718 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.957707 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:08.959292 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.959267 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-kube-api-access-jmtln" (OuterVolumeSpecName: "kube-api-access-jmtln") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "kube-api-access-jmtln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:13:08.959445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.959295 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:13:08.959445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.959433 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:08.959707 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:08.959678 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9e4b035-3b9a-4bd6-b199-85ea60214ca6" (UID: "d9e4b035-3b9a-4bd6-b199-85ea60214ca6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:13:09.058496 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.058460 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:09.058496 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.058490 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:09.058496 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.058505 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:09.058761 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.058516 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:09.058761 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.058525 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmtln\" (UniqueName: \"kubernetes.io/projected/d9e4b035-3b9a-4bd6-b199-85ea60214ca6-kube-api-access-jmtln\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:09.178169 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.178134 2560 generic.go:356] "Generic (PLEG): container finished" podID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerID="7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3" exitCode=0 Jul 24 16:13:09.178384 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.178181 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" Jul 24 16:13:09.178384 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.178168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" event={"ID":"d9e4b035-3b9a-4bd6-b199-85ea60214ca6","Type":"ContainerDied","Data":"7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3"} Jul 24 16:13:09.178384 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.178324 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp" event={"ID":"d9e4b035-3b9a-4bd6-b199-85ea60214ca6","Type":"ContainerDied","Data":"3005a0b5fbba09d40eef61c19184b11385fc064dbbdabbbd52f7bd07a85f5a39"} Jul 24 16:13:09.178384 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.178351 2560 scope.go:117] "RemoveContainer" containerID="7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3" Jul 24 16:13:09.189892 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.189750 2560 scope.go:117] "RemoveContainer" containerID="93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718" Jul 24 16:13:09.197375 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.197349 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp"] Jul 24 16:13:09.198708 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.198692 2560 scope.go:117] "RemoveContainer" containerID="7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3" Jul 24 16:13:09.199012 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:13:09.198991 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3\": container with ID starting with 7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3 not found: ID does not exist" containerID="7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3" Jul 24 16:13:09.199066 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.199022 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3"} err="failed to get container status \"7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3\": rpc error: code = NotFound desc = could not find container \"7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3\": container with ID starting with 7c3539a0f74a56a5b6c655da0d6ad03a21c9b456ba17a358be2ba63da5bbadb3 not found: ID does not exist" Jul 24 16:13:09.199066 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.199042 2560 scope.go:117] "RemoveContainer" containerID="93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718" Jul 24 16:13:09.199363 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:13:09.199339 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718\": container with ID starting with 93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718 not found: ID does not exist" containerID="93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718" Jul 24 16:13:09.199431 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.199373 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718"} err="failed to get container status \"93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718\": rpc error: code = NotFound desc = could not find container \"93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718\": container with ID starting with 93d98900618572617ea3c73141f52140df01fd997533714cd03babfcfbe63718 not found: ID does not exist" Jul 24 16:13:09.203661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:09.203640 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e03b82-openshift-default-977f5c98b-26xgp"] Jul 24 16:13:11.141378 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:11.141343 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" path="/var/lib/kubelet/pods/d9e4b035-3b9a-4bd6-b199-85ea60214ca6/volumes" Jul 24 16:13:14.491427 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:14.491383 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:14.571828 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:14.571793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:17.724837 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:17.724741 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:13:19.038884 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:19.038842 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd"] Jul 24 16:13:19.044359 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:19.044325 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-aedd03-j5lgd"] Jul 24 16:13:19.056396 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:19.056354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:19.141927 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:19.141880 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4c9cac2-a9a4-4854-94ea-44d1033dc381" path="/var/lib/kubelet/pods/e4c9cac2-a9a4-4854-94ea-44d1033dc381/volumes" Jul 24 16:13:19.488088 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:19.488005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:27.031537 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:27.031456 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:27.128809 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:27.128768 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:27.388843 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:27.388750 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w"] Jul 24 16:13:27.389033 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:27.388975 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" containerID="cri-o://d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f" gracePeriod=30 Jul 24 16:13:27.420058 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:27.420016 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:27.673487 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:27.673386 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:28.055012 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.054973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:28.460656 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.460579 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c"] Jul 24 16:13:28.460964 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.460951 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" Jul 24 16:13:28.461009 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.460965 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" Jul 24 16:13:28.461009 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.460988 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c9cac2-a9a4-4854-94ea-44d1033dc381" containerName="revoke-keys" Jul 24 16:13:28.461009 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.460993 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c9cac2-a9a4-4854-94ea-44d1033dc381" containerName="revoke-keys" Jul 24 16:13:28.461104 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.461050 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" Jul 24 16:13:28.461104 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.461060 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c9cac2-a9a4-4854-94ea-44d1033dc381" containerName="revoke-keys" Jul 24 16:13:28.463794 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.463778 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.467367 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.467345 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-57ceec-openshift-default-dockercfg-x559z\"" Jul 24 16:13:28.493952 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.493916 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c"] Jul 24 16:13:28.497051 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.497024 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:28.547873 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.547833 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-podinfo\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.547918 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f834cf88-1caa-44f9-8e38-d0cb37222976-istiod-ca-cert\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.547950 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-envoy\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.547995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-socket\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.548015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-token\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548184 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.548102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-credential-socket\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548184 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.548135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58jzf\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-kube-api-access-58jzf\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548184 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.548158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-certs\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.548184 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.548179 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-data\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.617749 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.617705 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:28.649440 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-token\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-credential-socket\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649500 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58jzf\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-kube-api-access-58jzf\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-certs\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-data\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-podinfo\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f834cf88-1caa-44f9-8e38-d0cb37222976-istiod-ca-cert\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-envoy\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.649848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.649834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-socket\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.650090 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.650015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-credential-socket\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.650090 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.650064 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-certs\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.650160 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.650096 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-data\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.650160 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.650151 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-socket\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.650532 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.650484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f834cf88-1caa-44f9-8e38-d0cb37222976-istiod-ca-cert\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.652148 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.652126 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-envoy\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.652401 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.652386 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-podinfo\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.664249 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.664220 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-token\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.665257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.665236 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58jzf\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-kube-api-access-58jzf\") pod \"e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.774709 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.774663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:28.957791 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.957762 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c"] Jul 24 16:13:28.960445 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:13:28.960406 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf834cf88_1caa_44f9_8e38_d0cb37222976.slice/crio-6bccd8bfcbf9ec2c88befdad2e42e350e6e4cb841c76049dfc208da5914459b1 WatchSource:0}: Error finding container 6bccd8bfcbf9ec2c88befdad2e42e350e6e4cb841c76049dfc208da5914459b1: Status 404 returned error can't find the container with id 6bccd8bfcbf9ec2c88befdad2e42e350e6e4cb841c76049dfc208da5914459b1 Jul 24 16:13:28.962954 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.962911 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:28.963057 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.963002 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:28.963057 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:28.963044 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:29.285224 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:29.285138 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" event={"ID":"f834cf88-1caa-44f9-8e38-d0cb37222976","Type":"ContainerStarted","Data":"5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79"} Jul 24 16:13:29.285224 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:29.285194 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" event={"ID":"f834cf88-1caa-44f9-8e38-d0cb37222976","Type":"ContainerStarted","Data":"6bccd8bfcbf9ec2c88befdad2e42e350e6e4cb841c76049dfc208da5914459b1"} Jul 24 16:13:29.311457 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:29.311381 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podStartSLOduration=1.311359438 podStartE2EDuration="1.311359438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:29.308535562 +0000 UTC m=+2232.731136616" watchObservedRunningTime="2026-07-24 16:13:29.311359438 +0000 UTC m=+2232.733960469" Jul 24 16:13:29.775667 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:29.775630 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:30.779799 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:30.779759 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:31.295365 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:31.295336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:31.296239 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:31.296222 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:31.396676 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:31.396643 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:31.514951 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:31.514916 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:31.616670 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:31.616588 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:31.781577 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:31.781537 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:32.596030 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.595993 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:32.654332 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.654257 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:13:32.711274 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.711242 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:32.791254 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791223 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-kube-api-access-8cxjl\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791337 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a106356-f753-4f80-807a-8df96cff5eb3-istio-podinfo\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791368 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-socket\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791395 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-data\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791433 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-istio-token\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791461 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-credential-socket\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791533 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-certs\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791569 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-envoy\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.791711 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791633 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a106356-f753-4f80-807a-8df96cff5eb3-istiod-ca-cert\") pod \"4a106356-f753-4f80-807a-8df96cff5eb3\" (UID: \"4a106356-f753-4f80-807a-8df96cff5eb3\") " Jul 24 16:13:32.792124 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.791868 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:32.792124 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.792009 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.792238 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.792166 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a106356-f753-4f80-807a-8df96cff5eb3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:13:32.792731 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.792612 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:32.794339 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.794195 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-kube-api-access-8cxjl" (OuterVolumeSpecName: "kube-api-access-8cxjl") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "kube-api-access-8cxjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:13:32.795581 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.794859 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:32.795581 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.794935 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:32.795581 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.795011 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:13:32.796983 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.796958 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a106356-f753-4f80-807a-8df96cff5eb3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:13:32.797246 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.797222 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a106356-f753-4f80-807a-8df96cff5eb3" (UID: "4a106356-f753-4f80-807a-8df96cff5eb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:13:32.892778 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892745 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.892778 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892771 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a106356-f753-4f80-807a-8df96cff5eb3-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.892778 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892780 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cxjl\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-kube-api-access-8cxjl\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.893020 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892790 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a106356-f753-4f80-807a-8df96cff5eb3-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.893020 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892799 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.893020 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892807 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.893020 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892814 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a106356-f753-4f80-807a-8df96cff5eb3-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:32.893020 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:32.892821 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a106356-f753-4f80-807a-8df96cff5eb3-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:33.307475 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.307439 2560 generic.go:356] "Generic (PLEG): container finished" podID="4a106356-f753-4f80-807a-8df96cff5eb3" containerID="d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f" exitCode=0 Jul 24 16:13:33.307645 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.307491 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" Jul 24 16:13:33.307645 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.307536 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" event={"ID":"4a106356-f753-4f80-807a-8df96cff5eb3","Type":"ContainerDied","Data":"d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f"} Jul 24 16:13:33.307645 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.307604 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w" event={"ID":"4a106356-f753-4f80-807a-8df96cff5eb3","Type":"ContainerDied","Data":"c5223d45f0316b6ff8cc5fe43b00aa676ea6b20e345c6e5ad4d6df0d46fac392"} Jul 24 16:13:33.307645 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.307629 2560 scope.go:117] "RemoveContainer" containerID="d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f" Jul 24 16:13:33.316643 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.316619 2560 scope.go:117] "RemoveContainer" containerID="f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f" Jul 24 16:13:33.324696 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.324672 2560 scope.go:117] "RemoveContainer" containerID="d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f" Jul 24 16:13:33.324957 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:13:33.324935 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f\": container with ID starting with d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f not found: ID does not exist" containerID="d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f" Jul 24 16:13:33.325004 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.324968 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f"} err="failed to get container status \"d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f\": rpc error: code = NotFound desc = could not find container \"d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f\": container with ID starting with d77bf5c675da723d8aba129f13dec283b8376e8310671d4952e30f80e6750b6f not found: ID does not exist" Jul 24 16:13:33.325004 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.324985 2560 scope.go:117] "RemoveContainer" containerID="f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f" Jul 24 16:13:33.325240 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:13:33.325222 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f\": container with ID starting with f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f not found: ID does not exist" containerID="f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f" Jul 24 16:13:33.325277 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.325247 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f"} err="failed to get container status \"f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f\": rpc error: code = NotFound desc = could not find container \"f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f\": container with ID starting with f8b1d1ae4abf4db6bb8cbb0e4694c4f2d585fbeac53949432aeb09a44a974c3f not found: ID does not exist" Jul 24 16:13:33.334243 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.334219 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w"] Jul 24 16:13:33.370833 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:33.365234 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-aedd03-openshift-default-c9c898954-7gd6w"] Jul 24 16:13:35.141534 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:35.141496 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" path="/var/lib/kubelet/pods/4a106356-f753-4f80-807a-8df96cff5eb3/volumes" Jul 24 16:13:35.941284 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:35.941249 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:36.429504 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.429465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:36.836281 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836247 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4"] Jul 24 16:13:36.836698 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836683 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" Jul 24 16:13:36.836746 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836699 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" Jul 24 16:13:36.836746 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836712 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" Jul 24 16:13:36.836746 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836718 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" Jul 24 16:13:36.836746 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836728 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" Jul 24 16:13:36.836746 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836733 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" Jul 24 16:13:36.836916 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836814 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" Jul 24 16:13:36.836916 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836822 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a106356-f753-4f80-807a-8df96cff5eb3" containerName="istio-proxy" Jul 24 16:13:36.836916 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.836831 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e4b035-3b9a-4bd6-b199-85ea60214ca6" containerName="istio-proxy" Jul 24 16:13:36.841443 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.841412 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.845257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.845233 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-afdeb1-openshift-default-dockercfg-qd44d\"" Jul 24 16:13:36.875051 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.875015 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4"] Jul 24 16:13:36.913388 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.913346 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:36.934819 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.934725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-socket\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.934996 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.934822 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-podinfo\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.934996 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.934855 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-token\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.934996 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.934888 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-data\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.934996 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.934951 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfngs\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-kube-api-access-gfngs\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.935249 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.935030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istiod-ca-cert\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.935249 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.935086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-certs\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.935249 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.935115 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-credential-socket\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:36.935249 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:36.935157 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-envoy\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.031520 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.031480 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:37.036882 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.036842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-podinfo\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037088 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.036902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-token\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037088 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.036955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-data\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037088 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.036996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfngs\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-kube-api-access-gfngs\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037088 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.037084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istiod-ca-cert\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037344 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.037140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-certs\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037344 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.037170 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-credential-socket\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037344 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.037219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-envoy\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037344 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.037276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-socket\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.037844 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.037815 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-socket\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.038450 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.038424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-data\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.038584 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.038493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-certs\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.038584 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.038499 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istiod-ca-cert\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.038743 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.038690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-credential-socket\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.041555 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.041526 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-envoy\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.041890 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.041862 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-podinfo\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.051153 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.051125 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-token\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.051693 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.051676 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfngs\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-kube-api-access-gfngs\") pod \"e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.152357 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.152239 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:37.301338 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.301217 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4"] Jul 24 16:13:37.304159 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:13:37.304133 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a1ae7b_217c_4d99_907c_aa9c73b9303d.slice/crio-86a1bc8905be54347dcbefa366a67886b1a5b3efac7025efdb9872437dd7eb1f WatchSource:0}: Error finding container 86a1bc8905be54347dcbefa366a67886b1a5b3efac7025efdb9872437dd7eb1f: Status 404 returned error can't find the container with id 86a1bc8905be54347dcbefa366a67886b1a5b3efac7025efdb9872437dd7eb1f Jul 24 16:13:37.306256 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.306224 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:37.306359 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.306288 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:37.306359 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.306339 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:37.330249 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:37.330222 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" event={"ID":"a2a1ae7b-217c-4d99-907c-aa9c73b9303d","Type":"ContainerStarted","Data":"86a1bc8905be54347dcbefa366a67886b1a5b3efac7025efdb9872437dd7eb1f"} Jul 24 16:13:38.337481 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:38.337446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" event={"ID":"a2a1ae7b-217c-4d99-907c-aa9c73b9303d","Type":"ContainerStarted","Data":"9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2"} Jul 24 16:13:38.395000 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:38.394938 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" podStartSLOduration=2.394916789 podStartE2EDuration="2.394916789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:38.39364232 +0000 UTC m=+2241.816243371" watchObservedRunningTime="2026-07-24 16:13:38.394916789 +0000 UTC m=+2241.817517884" Jul 24 16:13:39.152954 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:39.152913 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:39.157969 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:39.157939 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:39.342402 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:39.342366 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:39.343392 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:39.343374 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:13:40.487729 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:40.487699 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:40.624989 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:40.624942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:40.630231 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:40.630200 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:13:40.728187 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:40.728137 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:40.876609 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:40.876578 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:41.368892 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:41.368855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:41.469404 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:41.469372 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:45.016108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:45.016069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:13:47.337502 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:47.337461 2560 patch_prober.go:28] interesting pod/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 16:13:47.337885 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:47.337548 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 16:13:49.412725 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:49.412629 2560 generic.go:356] "Generic (PLEG): container finished" podID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerID="5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79" exitCode=0 Jul 24 16:13:49.412725 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:49.412705 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" event={"ID":"f834cf88-1caa-44f9-8e38-d0cb37222976","Type":"ContainerDied","Data":"5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79"} Jul 24 16:13:49.413186 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:49.413169 2560 scope.go:117] "RemoveContainer" containerID="5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79" Jul 24 16:13:49.414280 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:49.414248 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:49.414371 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:49.414355 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:49.414413 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:49.414393 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:50.421010 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:50.420974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" event={"ID":"f834cf88-1caa-44f9-8e38-d0cb37222976","Type":"ContainerStarted","Data":"492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2"} Jul 24 16:13:50.775653 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:50.775615 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:50.775868 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:50.775679 2560 patch_prober.go:28] interesting pod/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 16:13:50.775868 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:50.775744 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 16:13:51.776069 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:51.776030 2560 patch_prober.go:28] interesting pod/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 16:13:51.777137 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:51.776698 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 16:13:52.776203 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:52.776155 2560 patch_prober.go:28] interesting pod/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 16:13:52.776755 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:52.776235 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 16:13:53.583094 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.583046 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m"] Jul 24 16:13:53.587209 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.587176 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" Jul 24 16:13:53.593915 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.593891 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:13:53.594200 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.594174 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:13:53.611822 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.611782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a59643-d1c5-4934-9180-a47ea3b114fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m\" (UID: \"28a59643-d1c5-4934-9180-a47ea3b114fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" Jul 24 16:13:53.664818 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.664778 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m"] Jul 24 16:13:53.716685 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.715838 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a59643-d1c5-4934-9180-a47ea3b114fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m\" (UID: \"28a59643-d1c5-4934-9180-a47ea3b114fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" Jul 24 16:13:53.717152 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.716646 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a59643-d1c5-4934-9180-a47ea3b114fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m\" (UID: \"28a59643-d1c5-4934-9180-a47ea3b114fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" Jul 24 16:13:53.778890 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.778854 2560 patch_prober.go:28] interesting pod/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 16:13:53.779519 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.779471 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 16:13:53.904765 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:53.904660 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" Jul 24 16:13:54.147705 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:54.147674 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m"] Jul 24 16:13:54.442575 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:54.442492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" event={"ID":"28a59643-d1c5-4934-9180-a47ea3b114fd","Type":"ContainerStarted","Data":"b94f14688a849ba64b57cfc5e462c38656b4fd3a50c13f16a483ad5ce957e27c"} Jul 24 16:13:54.779726 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:54.779693 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:54.780210 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:54.779970 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:54.780859 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:54.780834 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:13:55.450082 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:55.449985 2560 generic.go:356] "Generic (PLEG): container finished" podID="28a59643-d1c5-4934-9180-a47ea3b114fd" containerID="3108e8f2d0fd3d56dc439f890a680db311e60e412019abffb117c5d3efe249aa" exitCode=0 Jul 24 16:13:55.450082 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:55.450067 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" event={"ID":"28a59643-d1c5-4934-9180-a47ea3b114fd","Type":"ContainerDied","Data":"3108e8f2d0fd3d56dc439f890a680db311e60e412019abffb117c5d3efe249aa"} Jul 24 16:13:56.600271 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:56.600245 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" Jul 24 16:13:56.649315 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:56.649264 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a59643-d1c5-4934-9180-a47ea3b114fd-maas-api-service-ca\") pod \"28a59643-d1c5-4934-9180-a47ea3b114fd\" (UID: \"28a59643-d1c5-4934-9180-a47ea3b114fd\") " Jul 24 16:13:56.649669 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:56.649642 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a59643-d1c5-4934-9180-a47ea3b114fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28a59643-d1c5-4934-9180-a47ea3b114fd" (UID: "28a59643-d1c5-4934-9180-a47ea3b114fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:13:56.751337 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:56.751242 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a59643-d1c5-4934-9180-a47ea3b114fd-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:13:57.464041 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:57.463956 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" Jul 24 16:13:57.464041 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:57.463980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m" event={"ID":"28a59643-d1c5-4934-9180-a47ea3b114fd","Type":"ContainerDied","Data":"b94f14688a849ba64b57cfc5e462c38656b4fd3a50c13f16a483ad5ce957e27c"} Jul 24 16:13:57.464266 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:57.464049 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b94f14688a849ba64b57cfc5e462c38656b4fd3a50c13f16a483ad5ce957e27c" Jul 24 16:13:57.508714 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:57.508678 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:13:58.469945 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:58.469908 2560 generic.go:356] "Generic (PLEG): container finished" podID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerID="9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2" exitCode=0 Jul 24 16:13:58.470469 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:58.469976 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" event={"ID":"a2a1ae7b-217c-4d99-907c-aa9c73b9303d","Type":"ContainerDied","Data":"9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2"} Jul 24 16:13:58.470469 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:58.470404 2560 scope.go:117] "RemoveContainer" containerID="9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2" Jul 24 16:13:58.471397 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:58.471368 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:58.471502 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:58.471449 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:58.471620 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:58.471498 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:13:59.481318 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:13:59.481257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" event={"ID":"a2a1ae7b-217c-4d99-907c-aa9c73b9303d","Type":"ContainerStarted","Data":"e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb"} Jul 24 16:14:00.152632 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:00.152592 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:14:00.152858 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:00.152831 2560 patch_prober.go:28] interesting pod/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 16:14:00.152907 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:00.152890 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 16:14:01.152623 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:01.152588 2560 patch_prober.go:28] interesting pod/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 16:14:01.153012 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:01.152647 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 16:14:02.153084 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:02.153043 2560 patch_prober.go:28] interesting pod/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 16:14:02.153510 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:02.153119 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 16:14:03.153270 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:03.153236 2560 patch_prober.go:28] interesting pod/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 24 16:14:03.153775 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:03.153292 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 24 16:14:04.156817 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:04.156787 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:14:04.157256 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:04.157077 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:14:04.157795 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:04.157776 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:14:08.925439 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:08.925401 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:10.371514 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:10.371476 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:13.665151 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:13.665108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:13.665853 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:13.665826 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m"] Jul 24 16:14:13.667898 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:13.667877 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-57ceec-fgb5m"] Jul 24 16:14:14.097558 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:14.097524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:14.394075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:14.393977 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:14:15.141097 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:15.141060 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a59643-d1c5-4934-9180-a47ea3b114fd" path="/var/lib/kubelet/pods/28a59643-d1c5-4934-9180-a47ea3b114fd/volumes" Jul 24 16:14:17.316857 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:17.316819 2560 scope.go:117] "RemoveContainer" containerID="5b53cbf1693b46478150ac2d9859e835e241e9392d05de2d43bc64e52b7a0c94" Jul 24 16:14:20.042560 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.042520 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:20.143533 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.143501 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:20.302966 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.302869 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c"] Jul 24 16:14:20.303139 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.303112 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" containerID="cri-o://492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2" gracePeriod=30 Jul 24 16:14:20.320158 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.320129 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:20.562334 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.562227 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:20.700096 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.700061 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg"] Jul 24 16:14:20.700477 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.700463 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a59643-d1c5-4934-9180-a47ea3b114fd" containerName="revoke-keys" Jul 24 16:14:20.700527 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.700480 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a59643-d1c5-4934-9180-a47ea3b114fd" containerName="revoke-keys" Jul 24 16:14:20.700572 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.700563 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a59643-d1c5-4934-9180-a47ea3b114fd" containerName="revoke-keys" Jul 24 16:14:20.703629 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.703611 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" Jul 24 16:14:20.706382 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.706352 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:14:20.706622 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.706605 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:14:20.712243 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.712222 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg"] Jul 24 16:14:20.802348 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.802279 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad85dff7-a857-4f52-a175-f5ec19da6a23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg\" (UID: \"ad85dff7-a857-4f52-a175-f5ec19da6a23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" Jul 24 16:14:20.903878 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.903771 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad85dff7-a857-4f52-a175-f5ec19da6a23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg\" (UID: \"ad85dff7-a857-4f52-a175-f5ec19da6a23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" Jul 24 16:14:20.904453 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:20.904432 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad85dff7-a857-4f52-a175-f5ec19da6a23-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg\" (UID: \"ad85dff7-a857-4f52-a175-f5ec19da6a23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" Jul 24 16:14:21.015921 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:21.015885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" Jul 24 16:14:21.157932 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:21.157903 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg"] Jul 24 16:14:21.160278 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:14:21.160246 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad85dff7_a857_4f52_a175_f5ec19da6a23.slice/crio-d905414dd8039466d514b11a07fac449cbf44b0aa91ae832bec40eb2ca09fd73 WatchSource:0}: Error finding container d905414dd8039466d514b11a07fac449cbf44b0aa91ae832bec40eb2ca09fd73: Status 404 returned error can't find the container with id d905414dd8039466d514b11a07fac449cbf44b0aa91ae832bec40eb2ca09fd73 Jul 24 16:14:21.605271 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:21.605233 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" event={"ID":"ad85dff7-a857-4f52-a175-f5ec19da6a23","Type":"ContainerStarted","Data":"81b41d2e1b66788ac2c927f9cd5635ef10f66c3f1fa7835a8289866e31837a85"} Jul 24 16:14:21.605477 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:21.605297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" event={"ID":"ad85dff7-a857-4f52-a175-f5ec19da6a23","Type":"ContainerStarted","Data":"d905414dd8039466d514b11a07fac449cbf44b0aa91ae832bec40eb2ca09fd73"} Jul 24 16:14:21.640728 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:21.640668 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" podStartSLOduration=1.640644493 podStartE2EDuration="1.640644493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:21.639889268 +0000 UTC m=+2285.062490318" watchObservedRunningTime="2026-07-24 16:14:21.640644493 +0000 UTC m=+2285.063245525" Jul 24 16:14:22.612205 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:22.612171 2560 generic.go:356] "Generic (PLEG): container finished" podID="ad85dff7-a857-4f52-a175-f5ec19da6a23" containerID="81b41d2e1b66788ac2c927f9cd5635ef10f66c3f1fa7835a8289866e31837a85" exitCode=0 Jul 24 16:14:22.612665 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:22.612250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" event={"ID":"ad85dff7-a857-4f52-a175-f5ec19da6a23","Type":"ContainerDied","Data":"81b41d2e1b66788ac2c927f9cd5635ef10f66c3f1fa7835a8289866e31837a85"} Jul 24 16:14:23.750825 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:23.750799 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" Jul 24 16:14:23.934618 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:23.934532 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad85dff7-a857-4f52-a175-f5ec19da6a23-maas-api-service-ca\") pod \"ad85dff7-a857-4f52-a175-f5ec19da6a23\" (UID: \"ad85dff7-a857-4f52-a175-f5ec19da6a23\") " Jul 24 16:14:23.934935 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:23.934911 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad85dff7-a857-4f52-a175-f5ec19da6a23-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad85dff7-a857-4f52-a175-f5ec19da6a23" (UID: "ad85dff7-a857-4f52-a175-f5ec19da6a23"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:14:24.036173 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:24.036135 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad85dff7-a857-4f52-a175-f5ec19da6a23-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:24.623503 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:24.623474 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" Jul 24 16:14:24.623676 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:24.623471 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg" event={"ID":"ad85dff7-a857-4f52-a175-f5ec19da6a23","Type":"ContainerDied","Data":"d905414dd8039466d514b11a07fac449cbf44b0aa91ae832bec40eb2ca09fd73"} Jul 24 16:14:24.623676 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:24.623605 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d905414dd8039466d514b11a07fac449cbf44b0aa91ae832bec40eb2ca09fd73" Jul 24 16:14:24.783134 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:24.783094 2560 patch_prober.go:28] interesting pod/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 16:14:24.783519 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:24.783180 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 16:14:25.566319 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.566280 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:14:25.630435 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.630343 2560 generic.go:356] "Generic (PLEG): container finished" podID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerID="492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2" exitCode=0 Jul 24 16:14:25.630435 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.630384 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" Jul 24 16:14:25.630435 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.630421 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" event={"ID":"f834cf88-1caa-44f9-8e38-d0cb37222976","Type":"ContainerDied","Data":"492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2"} Jul 24 16:14:25.630673 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.630472 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c" event={"ID":"f834cf88-1caa-44f9-8e38-d0cb37222976","Type":"ContainerDied","Data":"6bccd8bfcbf9ec2c88befdad2e42e350e6e4cb841c76049dfc208da5914459b1"} Jul 24 16:14:25.630673 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.630497 2560 scope.go:117] "RemoveContainer" containerID="492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2" Jul 24 16:14:25.639332 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.639291 2560 scope.go:117] "RemoveContainer" containerID="5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79" Jul 24 16:14:25.647567 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.647550 2560 scope.go:117] "RemoveContainer" containerID="492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2" Jul 24 16:14:25.647832 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:14:25.647812 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2\": container with ID starting with 492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2 not found: ID does not exist" containerID="492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2" Jul 24 16:14:25.647886 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.647842 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2"} err="failed to get container status \"492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2\": rpc error: code = NotFound desc = could not find container \"492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2\": container with ID starting with 492d752c744f09061672221a9ae32f92462853a2975210a0d8d2bedee9fdc8b2 not found: ID does not exist" Jul 24 16:14:25.647886 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.647859 2560 scope.go:117] "RemoveContainer" containerID="5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79" Jul 24 16:14:25.648089 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:14:25.648071 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79\": container with ID starting with 5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79 not found: ID does not exist" containerID="5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79" Jul 24 16:14:25.648146 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.648098 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79"} err="failed to get container status \"5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79\": rpc error: code = NotFound desc = could not find container \"5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79\": container with ID starting with 5184ed765129edd42201c5259fdd09c65b671fc41a616bff8ff5d7677e352c79 not found: ID does not exist" Jul 24 16:14:25.750379 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750349 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58jzf\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-kube-api-access-58jzf\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750570 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750444 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-envoy\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750570 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750485 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f834cf88-1caa-44f9-8e38-d0cb37222976-istiod-ca-cert\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750570 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750524 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-certs\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750570 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750556 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-podinfo\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750602 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-token\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750661 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-credential-socket\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750687 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-data\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.750801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750750 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-socket\") pod \"f834cf88-1caa-44f9-8e38-d0cb37222976\" (UID: \"f834cf88-1caa-44f9-8e38-d0cb37222976\") " Jul 24 16:14:25.751002 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750865 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:25.751002 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.750916 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f834cf88-1caa-44f9-8e38-d0cb37222976-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:14:25.751102 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.751042 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:25.751102 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.751072 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:25.751380 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.751292 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f834cf88-1caa-44f9-8e38-d0cb37222976-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.751380 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.751340 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.751380 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.751355 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.751380 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.751371 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.751789 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.751757 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-data" (OuterVolumeSpecName: "istio-data") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:25.752986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.752961 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-kube-api-access-58jzf" (OuterVolumeSpecName: "kube-api-access-58jzf") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "kube-api-access-58jzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:14:25.753372 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.753344 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:25.753372 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.753342 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:14:25.753526 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.753506 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-token" (OuterVolumeSpecName: "istio-token") pod "f834cf88-1caa-44f9-8e38-d0cb37222976" (UID: "f834cf88-1caa-44f9-8e38-d0cb37222976"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:14:25.852038 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.851993 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.852038 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.852024 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.852038 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.852033 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.852038 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.852041 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f834cf88-1caa-44f9-8e38-d0cb37222976-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.852528 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.852049 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58jzf\" (UniqueName: \"kubernetes.io/projected/f834cf88-1caa-44f9-8e38-d0cb37222976-kube-api-access-58jzf\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:25.953475 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.953445 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c"] Jul 24 16:14:25.958006 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:25.957980 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-57ceec-openshift-default-8476c59664-8bm2c"] Jul 24 16:14:27.140895 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:27.140861 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" path="/var/lib/kubelet/pods/f834cf88-1caa-44f9-8e38-d0cb37222976/volumes" Jul 24 16:14:27.458564 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:27.458479 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:36.221180 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:36.221143 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:36.322342 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:36.322288 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:36.422954 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:36.422917 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:40.782318 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:40.782265 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg"] Jul 24 16:14:40.791573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:40.791526 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-afdeb1-sqnhg"] Jul 24 16:14:40.798986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:40.798950 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:40.809925 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:40.809884 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:14:41.142021 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:41.141933 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad85dff7-a857-4f52-a175-f5ec19da6a23" path="/var/lib/kubelet/pods/ad85dff7-a857-4f52-a175-f5ec19da6a23/volumes" Jul 24 16:14:41.197530 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:41.197495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:47.018149 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:47.018113 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:47.117013 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:47.116975 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:47.291173 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:47.291050 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4"] Jul 24 16:14:47.291662 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:47.291604 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" containerID="cri-o://e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb" gracePeriod=30 Jul 24 16:14:47.336371 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:47.333778 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:47.592284 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:47.592186 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:47.962486 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:47.962406 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:48.256385 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.256262 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296"] Jul 24 16:14:48.256853 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.256818 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" Jul 24 16:14:48.256853 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.256839 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" Jul 24 16:14:48.256986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.256892 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" Jul 24 16:14:48.256986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.256901 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" Jul 24 16:14:48.256986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.256917 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad85dff7-a857-4f52-a175-f5ec19da6a23" containerName="revoke-keys" Jul 24 16:14:48.256986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.256925 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad85dff7-a857-4f52-a175-f5ec19da6a23" containerName="revoke-keys" Jul 24 16:14:48.257199 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.257002 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" Jul 24 16:14:48.257199 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.257012 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f834cf88-1caa-44f9-8e38-d0cb37222976" containerName="istio-proxy" Jul 24 16:14:48.257199 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.257020 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad85dff7-a857-4f52-a175-f5ec19da6a23" containerName="revoke-keys" Jul 24 16:14:48.261889 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.261861 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.264241 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.264221 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-fafc28-openshift-default-dockercfg-qtpcj\"" Jul 24 16:14:48.296086 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.296054 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:48.332799 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.332767 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296"] Jul 24 16:14:48.362443 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.362403 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-data\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.362818 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.362791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-certs\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.363016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.362999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882a76ba-6c28-4231-8080-6dab2bee7399-istiod-ca-cert\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.363188 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.363172 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-socket\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.363364 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.363347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-istio-token\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.363534 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.363518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-envoy\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.363725 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.363708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8bvm\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-kube-api-access-w8bvm\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.363876 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.363861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882a76ba-6c28-4231-8080-6dab2bee7399-istio-podinfo\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.364020 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.364004 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-credential-socket\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.410857 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.410816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:48.465693 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.465647 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882a76ba-6c28-4231-8080-6dab2bee7399-istio-podinfo\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.465949 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.465928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-credential-socket\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.466079 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.466064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-data\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.466245 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.466227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-certs\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.466410 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.466391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882a76ba-6c28-4231-8080-6dab2bee7399-istiod-ca-cert\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.466555 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.466534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-socket\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.466704 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.466681 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-credential-socket\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.466830 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.466696 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-istio-token\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.466933 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.466907 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-certs\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.467045 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.467029 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-envoy\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.467206 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.467182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8bvm\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-kube-api-access-w8bvm\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.467553 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.467093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-socket\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.467698 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.467641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882a76ba-6c28-4231-8080-6dab2bee7399-istiod-ca-cert\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.467976 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.467946 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-data\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.469818 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.469791 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-envoy\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.470069 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.470046 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882a76ba-6c28-4231-8080-6dab2bee7399-istio-podinfo\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.475234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.475206 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-istio-token\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.475385 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.475367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8bvm\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-kube-api-access-w8bvm\") pod \"e2e-shared-a-fafc28-openshift-default-865967cd4-45296\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.575031 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.574989 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:48.713697 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.713670 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296"] Jul 24 16:14:48.715927 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:14:48.715900 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882a76ba_6c28_4231_8080_6dab2bee7399.slice/crio-faafd7ebb849ceb54cfa301d0c1ae7f0e81558d78448f514c1a46cd12ccd342e WatchSource:0}: Error finding container faafd7ebb849ceb54cfa301d0c1ae7f0e81558d78448f514c1a46cd12ccd342e: Status 404 returned error can't find the container with id faafd7ebb849ceb54cfa301d0c1ae7f0e81558d78448f514c1a46cd12ccd342e Jul 24 16:14:48.718155 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.718123 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:14:48.718243 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.718189 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:14:48.718243 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.718216 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:14:48.750498 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:48.750461 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" event={"ID":"882a76ba-6c28-4231-8080-6dab2bee7399","Type":"ContainerStarted","Data":"faafd7ebb849ceb54cfa301d0c1ae7f0e81558d78448f514c1a46cd12ccd342e"} Jul 24 16:14:49.159329 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:49.159276 2560 patch_prober.go:28] interesting pod/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 16:14:49.159495 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:49.159355 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 16:14:49.757673 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:49.757634 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" event={"ID":"882a76ba-6c28-4231-8080-6dab2bee7399","Type":"ContainerStarted","Data":"e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd"} Jul 24 16:14:49.784026 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:49.783973 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" podStartSLOduration=1.78395621 podStartE2EDuration="1.78395621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:49.781866208 +0000 UTC m=+2313.204467252" watchObservedRunningTime="2026-07-24 16:14:49.78395621 +0000 UTC m=+2313.206557298" Jul 24 16:14:50.575351 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:50.575289 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:50.580319 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:50.580277 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:50.763410 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:50.763377 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:50.764412 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:50.764391 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:14:51.778552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:51.778520 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:51.868460 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:51.868423 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:51.944465 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:51.944426 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:52.062265 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.062166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:52.549713 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.549675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:52.556105 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.556084 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:14:52.626106 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.626024 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:52.708748 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708714 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-credential-socket\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.708937 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708769 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-data\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.708937 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708802 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-socket\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.708937 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708837 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-token\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.708937 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708854 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-envoy\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.708937 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708886 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-podinfo\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.708937 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708923 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfngs\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-kube-api-access-gfngs\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.709260 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.708967 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-certs\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.709260 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709011 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istiod-ca-cert\") pod \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\" (UID: \"a2a1ae7b-217c-4d99-907c-aa9c73b9303d\") " Jul 24 16:14:52.709260 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709023 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:52.709260 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709170 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:52.709506 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709432 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.709506 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709453 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.709607 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709504 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:14:52.709607 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709585 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:52.709779 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.709756 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:52.711573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.711539 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:14:52.711793 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.711768 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:14:52.711866 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.711814 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-kube-api-access-gfngs" (OuterVolumeSpecName: "kube-api-access-gfngs") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "kube-api-access-gfngs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:14:52.711941 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.711924 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2a1ae7b-217c-4d99-907c-aa9c73b9303d" (UID: "a2a1ae7b-217c-4d99-907c-aa9c73b9303d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:14:52.777766 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.777725 2560 generic.go:356] "Generic (PLEG): container finished" podID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerID="e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb" exitCode=0 Jul 24 16:14:52.777963 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.777791 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" Jul 24 16:14:52.777963 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.777797 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" event={"ID":"a2a1ae7b-217c-4d99-907c-aa9c73b9303d","Type":"ContainerDied","Data":"e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb"} Jul 24 16:14:52.777963 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.777843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4" event={"ID":"a2a1ae7b-217c-4d99-907c-aa9c73b9303d","Type":"ContainerDied","Data":"86a1bc8905be54347dcbefa366a67886b1a5b3efac7025efdb9872437dd7eb1f"} Jul 24 16:14:52.777963 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.777865 2560 scope.go:117] "RemoveContainer" containerID="e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb" Jul 24 16:14:52.787416 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.787394 2560 scope.go:117] "RemoveContainer" containerID="9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2" Jul 24 16:14:52.795679 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.795662 2560 scope.go:117] "RemoveContainer" containerID="e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb" Jul 24 16:14:52.795968 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:14:52.795941 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb\": container with ID starting with e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb not found: ID does not exist" containerID="e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb" Jul 24 16:14:52.796094 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.795973 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb"} err="failed to get container status \"e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb\": rpc error: code = NotFound desc = could not find container \"e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb\": container with ID starting with e5bdb0cd1dbed1b5c7f8db6c021fd33118c684b2e5eb13cf260ed0f0ccc9affb not found: ID does not exist" Jul 24 16:14:52.796094 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.795993 2560 scope.go:117] "RemoveContainer" containerID="9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2" Jul 24 16:14:52.796680 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:14:52.796649 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2\": container with ID starting with 9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2 not found: ID does not exist" containerID="9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2" Jul 24 16:14:52.796779 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.796689 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2"} err="failed to get container status \"9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2\": rpc error: code = NotFound desc = could not find container \"9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2\": container with ID starting with 9ead71524b1ed8ccffdd98a7dd422b0bc6673287b523537d41d695edd5a051b2 not found: ID does not exist" Jul 24 16:14:52.798900 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.798879 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4"] Jul 24 16:14:52.802195 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.802168 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-afdeb1-openshift-default-bb9864fdd-nmkq4"] Jul 24 16:14:52.810613 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.810589 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.810613 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.810613 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.810747 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.810621 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.810747 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.810629 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.810747 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.810648 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfngs\" (UniqueName: \"kubernetes.io/projected/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-kube-api-access-gfngs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.810747 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.810660 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:52.810747 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:52.810668 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2a1ae7b-217c-4d99-907c-aa9c73b9303d-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:14:53.140587 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:53.140542 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" path="/var/lib/kubelet/pods/a2a1ae7b-217c-4d99-907c-aa9c73b9303d/volumes" Jul 24 16:14:56.316074 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.316006 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:56.687985 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.687888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:56.977252 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.977168 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4"] Jul 24 16:14:56.977621 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.977606 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" Jul 24 16:14:56.977675 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.977623 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" Jul 24 16:14:56.977675 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.977633 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" Jul 24 16:14:56.977675 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.977639 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" Jul 24 16:14:56.977772 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.977709 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" Jul 24 16:14:56.977772 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.977718 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2a1ae7b-217c-4d99-907c-aa9c73b9303d" containerName="istio-proxy" Jul 24 16:14:56.982611 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.982585 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:56.985353 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:56.985328 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c6c91e-openshift-default-dockercfg-2nfmk\"" Jul 24 16:14:57.012854 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.012826 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:57.051128 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051091 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-token\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051297 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051133 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-credential-socket\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051297 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-socket\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051297 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6a470b-c088-4ae5-9597-7eba021b8cb5-istiod-ca-cert\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051356 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmvh6\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-kube-api-access-bmvh6\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051380 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-podinfo\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051397 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-data\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051425 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-certs\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.051587 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.051483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-envoy\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.077698 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.077648 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4"] Jul 24 16:14:57.152737 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.152706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-credential-socket\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.152939 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.152748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-socket\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.152939 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.152885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6a470b-c088-4ae5-9597-7eba021b8cb5-istiod-ca-cert\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.152962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmvh6\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-kube-api-access-bmvh6\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.152984 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-podinfo\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-data\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153063 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-certs\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153287 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153080 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-envoy\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153287 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153116 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-token\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153287 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153139 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-socket\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153287 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153188 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-credential-socket\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153523 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-data\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153566 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153545 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6a470b-c088-4ae5-9597-7eba021b8cb5-istiod-ca-cert\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.153662 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.153645 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-certs\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.155958 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.155933 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-envoy\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.156074 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.156057 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-podinfo\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.165513 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.165485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmvh6\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-kube-api-access-bmvh6\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.165663 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.165485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-token\") pod \"e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.197460 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.197429 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:14:57.205871 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.205840 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:57.293984 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.293950 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:57.437773 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.437748 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4"] Jul 24 16:14:57.439957 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:14:57.439926 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6a470b_c088_4ae5_9597_7eba021b8cb5.slice/crio-371951e23cabcd88a05859efa83b02068697828aa946514a054eb5469e2dae3f WatchSource:0}: Error finding container 371951e23cabcd88a05859efa83b02068697828aa946514a054eb5469e2dae3f: Status 404 returned error can't find the container with id 371951e23cabcd88a05859efa83b02068697828aa946514a054eb5469e2dae3f Jul 24 16:14:57.442378 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.442343 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:14:57.442489 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.442429 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:14:57.442489 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.442480 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:14:57.806257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.806219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" event={"ID":"df6a470b-c088-4ae5-9597-7eba021b8cb5","Type":"ContainerStarted","Data":"b5028a4d9f61390d4e22b4e08b97ad2a99f905d0d7f1c0e6d3a9980a1d56e35a"} Jul 24 16:14:57.806478 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.806284 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" event={"ID":"df6a470b-c088-4ae5-9597-7eba021b8cb5","Type":"ContainerStarted","Data":"371951e23cabcd88a05859efa83b02068697828aa946514a054eb5469e2dae3f"} Jul 24 16:14:57.827689 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:57.827621 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" podStartSLOduration=1.8276015970000001 podStartE2EDuration="1.827601597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:57.82464814 +0000 UTC m=+2321.247249189" watchObservedRunningTime="2026-07-24 16:14:57.827601597 +0000 UTC m=+2321.250202627" Jul 24 16:14:58.295035 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:58.294985 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:58.300082 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:58.300055 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:58.811452 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:58.811419 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:58.812483 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:58.812459 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:14:59.773112 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:59.773060 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:14:59.874676 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:14:59.874565 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:00.149012 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.148930 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh"] Jul 24 16:15:00.152426 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.152409 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b"] Jul 24 16:15:00.152528 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.152511 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" Jul 24 16:15:00.155314 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.155276 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:15:00.155575 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.155560 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" Jul 24 16:15:00.180336 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.180281 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" Jul 24 16:15:00.184331 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.182079 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh"] Jul 24 16:15:00.191643 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.188819 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b"] Jul 24 16:15:00.201351 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.197076 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" Jul 24 16:15:00.370763 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.363841 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:00.392997 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.392950 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh"] Jul 24 16:15:00.395430 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:15:00.395370 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622850fb_3022_47fd_b57d_8e84978d4c0f.slice/crio-5b9925c60e124e4e40bc6ae62674a66e03633cdd48aff53d5b6638d658baaf2b WatchSource:0}: Error finding container 5b9925c60e124e4e40bc6ae62674a66e03633cdd48aff53d5b6638d658baaf2b: Status 404 returned error can't find the container with id 5b9925c60e124e4e40bc6ae62674a66e03633cdd48aff53d5b6638d658baaf2b Jul 24 16:15:00.469337 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.457925 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b"] Jul 24 16:15:00.484054 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:15:00.484021 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860566b7_7697_4722_b994_f25413231f03.slice/crio-94c2e0b884954182ab8e6bb4ebe19dcd67d69f1ef5770baf015ffbfce5adc67d WatchSource:0}: Error finding container 94c2e0b884954182ab8e6bb4ebe19dcd67d69f1ef5770baf015ffbfce5adc67d: Status 404 returned error can't find the container with id 94c2e0b884954182ab8e6bb4ebe19dcd67d69f1ef5770baf015ffbfce5adc67d Jul 24 16:15:00.589863 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.589830 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:00.824771 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.824733 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" event={"ID":"860566b7-7697-4722-b994-f25413231f03","Type":"ContainerStarted","Data":"77c396c401e71c221d1771fc0abb0ee74601e2378770d0983fe16bd54f10d4c2"} Jul 24 16:15:00.824940 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.824799 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" event={"ID":"860566b7-7697-4722-b994-f25413231f03","Type":"ContainerStarted","Data":"94c2e0b884954182ab8e6bb4ebe19dcd67d69f1ef5770baf015ffbfce5adc67d"} Jul 24 16:15:00.826538 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.826508 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" event={"ID":"622850fb-3022-47fd-b57d-8e84978d4c0f","Type":"ContainerStarted","Data":"a42ceb67888a21295ca71efb8fa5f0edebd8dafbebd89b39bd20f9ba35cb759e"} Jul 24 16:15:00.826688 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.826556 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" event={"ID":"622850fb-3022-47fd-b57d-8e84978d4c0f","Type":"ContainerStarted","Data":"5b9925c60e124e4e40bc6ae62674a66e03633cdd48aff53d5b6638d658baaf2b"} Jul 24 16:15:00.841825 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.841779 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" podStartSLOduration=0.841765606 podStartE2EDuration="841.765606ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:15:00.839980319 +0000 UTC m=+2324.262581349" watchObservedRunningTime="2026-07-24 16:15:00.841765606 +0000 UTC m=+2324.264366644" Jul 24 16:15:00.854578 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.854536 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" podStartSLOduration=0.854522472 podStartE2EDuration="854.522472ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:15:00.852791051 +0000 UTC m=+2324.275392096" watchObservedRunningTime="2026-07-24 16:15:00.854522472 +0000 UTC m=+2324.277123501" Jul 24 16:15:00.868015 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.867978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:00.956890 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:00.956799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:01.833447 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:01.833403 2560 generic.go:356] "Generic (PLEG): container finished" podID="622850fb-3022-47fd-b57d-8e84978d4c0f" containerID="a42ceb67888a21295ca71efb8fa5f0edebd8dafbebd89b39bd20f9ba35cb759e" exitCode=0 Jul 24 16:15:01.833636 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:01.833492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" event={"ID":"622850fb-3022-47fd-b57d-8e84978d4c0f","Type":"ContainerDied","Data":"a42ceb67888a21295ca71efb8fa5f0edebd8dafbebd89b39bd20f9ba35cb759e"} Jul 24 16:15:02.841476 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:02.841386 2560 generic.go:356] "Generic (PLEG): container finished" podID="860566b7-7697-4722-b994-f25413231f03" containerID="77c396c401e71c221d1771fc0abb0ee74601e2378770d0983fe16bd54f10d4c2" exitCode=0 Jul 24 16:15:02.841476 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:02.841464 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" event={"ID":"860566b7-7697-4722-b994-f25413231f03","Type":"ContainerDied","Data":"77c396c401e71c221d1771fc0abb0ee74601e2378770d0983fe16bd54f10d4c2"} Jul 24 16:15:02.983931 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:02.983907 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" Jul 24 16:15:03.852801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:03.849477 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" Jul 24 16:15:03.852801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:03.849531 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-m77lh" event={"ID":"622850fb-3022-47fd-b57d-8e84978d4c0f","Type":"ContainerDied","Data":"5b9925c60e124e4e40bc6ae62674a66e03633cdd48aff53d5b6638d658baaf2b"} Jul 24 16:15:03.852801 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:03.849581 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b9925c60e124e4e40bc6ae62674a66e03633cdd48aff53d5b6638d658baaf2b" Jul 24 16:15:04.005544 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:04.005516 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" Jul 24 16:15:04.303590 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:04.303555 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:04.856149 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:04.856121 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" Jul 24 16:15:04.856569 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:04.856118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b" event={"ID":"860566b7-7697-4722-b994-f25413231f03","Type":"ContainerDied","Data":"94c2e0b884954182ab8e6bb4ebe19dcd67d69f1ef5770baf015ffbfce5adc67d"} Jul 24 16:15:04.856569 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:04.856265 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94c2e0b884954182ab8e6bb4ebe19dcd67d69f1ef5770baf015ffbfce5adc67d" Jul 24 16:15:05.764973 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.764937 2560 patch_prober.go:28] interesting pod/e2e-shared-a-fafc28-openshift-default-865967cd4-45296 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 16:15:05.765615 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.765277 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 16:15:05.865153 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.865049 2560 generic.go:356] "Generic (PLEG): container finished" podID="882a76ba-6c28-4231-8080-6dab2bee7399" containerID="e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd" exitCode=0 Jul 24 16:15:05.865664 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.865165 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" event={"ID":"882a76ba-6c28-4231-8080-6dab2bee7399","Type":"ContainerDied","Data":"e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd"} Jul 24 16:15:05.865739 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.865716 2560 scope.go:117] "RemoveContainer" containerID="e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd" Jul 24 16:15:05.867410 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.867369 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:15:05.867518 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.867462 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:15:05.867518 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:05.867510 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:15:06.876084 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:06.876037 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" event={"ID":"882a76ba-6c28-4231-8080-6dab2bee7399","Type":"ContainerStarted","Data":"7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449"} Jul 24 16:15:07.575877 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:07.575839 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:15:07.576378 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:07.576354 2560 patch_prober.go:28] interesting pod/e2e-shared-a-fafc28-openshift-default-865967cd4-45296 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 16:15:07.576561 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:07.576535 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 16:15:08.576283 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:08.576232 2560 patch_prober.go:28] interesting pod/e2e-shared-a-fafc28-openshift-default-865967cd4-45296 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 16:15:08.576808 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:08.576323 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 16:15:09.576695 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:09.576650 2560 patch_prober.go:28] interesting pod/e2e-shared-a-fafc28-openshift-default-865967cd4-45296 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 16:15:09.577169 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:09.576719 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 16:15:10.575962 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:10.575919 2560 patch_prober.go:28] interesting pod/e2e-shared-a-fafc28-openshift-default-865967cd4-45296 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 16:15:10.576275 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:10.576234 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 16:15:10.909333 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:10.909237 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:15:11.579764 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:11.579734 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:15:11.580026 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:11.580005 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:15:11.580763 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:11.580742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:15:12.923578 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:12.923539 2560 generic.go:356] "Generic (PLEG): container finished" podID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerID="b5028a4d9f61390d4e22b4e08b97ad2a99f905d0d7f1c0e6d3a9980a1d56e35a" exitCode=0 Jul 24 16:15:12.924007 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:12.923615 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" event={"ID":"df6a470b-c088-4ae5-9597-7eba021b8cb5","Type":"ContainerDied","Data":"b5028a4d9f61390d4e22b4e08b97ad2a99f905d0d7f1c0e6d3a9980a1d56e35a"} Jul 24 16:15:12.924187 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:12.924172 2560 scope.go:117] "RemoveContainer" containerID="b5028a4d9f61390d4e22b4e08b97ad2a99f905d0d7f1c0e6d3a9980a1d56e35a" Jul 24 16:15:12.925225 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:12.925187 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:15:12.925356 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:12.925274 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:15:12.925356 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:12.925334 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:15:13.811888 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:13.811833 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:15:13.931549 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:13.931503 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" event={"ID":"df6a470b-c088-4ae5-9597-7eba021b8cb5","Type":"ContainerStarted","Data":"883011595cc9a82fb347b0b2f5d1b0083bccbdc373363ac24c6dd8b4b941801d"} Jul 24 16:15:14.294842 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:14.294800 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:15:14.295981 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:14.295953 2560 patch_prober.go:28] interesting pod/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 16:15:14.296120 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:14.296023 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 16:15:15.294781 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:15.294743 2560 patch_prober.go:28] interesting pod/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 16:15:15.295149 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:15.294817 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 16:15:16.295144 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:16.295107 2560 patch_prober.go:28] interesting pod/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 16:15:16.295694 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:16.295174 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 16:15:17.294674 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:17.294627 2560 patch_prober.go:28] interesting pod/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 16:15:17.294890 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:17.294705 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 16:15:17.342388 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:17.342356 2560 scope.go:117] "RemoveContainer" containerID="78f12fd026729ff62ca9859b611159c95ca5abac3f201f9bfa5a19f9830e94a9" Jul 24 16:15:18.298543 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:18.298516 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:15:18.298804 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:18.298782 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:15:18.299436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:18.299418 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:15:20.057006 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:20.056939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:25.200214 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:25.200176 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:15:26.799544 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:26.799503 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:26.897780 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:26.897711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:53.335761 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:53.335727 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:15:57.302002 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:15:57.301968 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:16:12.847460 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:16:12.847428 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:16:32.993493 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:16:32.993455 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:16:52.997681 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:16:52.997587 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:17:12.670552 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:17:12.670512 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:17:28.121513 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:17:28.121473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:17:32.568688 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:17:32.568647 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:17:42.619857 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:17:42.619818 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:17:47.899950 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:17:47.899915 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:18:04.509070 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:04.509030 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:18:05.505409 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:05.505373 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:18:08.013896 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:08.013808 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:18:09.054089 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:09.054057 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:18:24.101655 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:24.101614 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:18:32.003072 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:32.003036 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:18:36.737700 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:36.737663 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:18:56.506633 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:18:56.506600 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:19:09.504729 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:09.504684 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:19:09.572049 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:09.572016 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:19:17.131850 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:17.131816 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:19:17.400546 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:17.400452 2560 scope.go:117] "RemoveContainer" containerID="741cc1b3d80c85df33372c2933c1f99426381e7702ab788599cae451a547c5d0" Jul 24 16:19:17.409291 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:17.409272 2560 scope.go:117] "RemoveContainer" containerID="f44e0e6650218b34c42a37f383d60f3143c41c3836abea54559aa080ebd56b11" Jul 24 16:19:25.320724 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:25.320688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:19:26.751516 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:26.751467 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:19:39.072178 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:39.072093 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:19:43.881232 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:43.881190 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:19:49.918956 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:19:49.918917 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:20:03.057914 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:03.057873 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:20:07.114608 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:07.114569 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:20:12.998271 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:12.998238 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:20:13.160050 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:13.160013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:20:17.433749 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:17.433712 2560 scope.go:117] "RemoveContainer" containerID="3108e8f2d0fd3d56dc439f890a680db311e60e412019abffb117c5d3efe249aa" Jul 24 16:20:22.799970 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:22.799932 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:20:48.704657 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:48.704619 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:20:52.730329 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:52.730273 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:20:53.852431 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:20:53.852388 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:05.165044 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:05.164999 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:05.266514 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:05.266467 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:05.603639 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:05.603604 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:21:13.439261 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.439174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:13.757164 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.757079 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8"] Jul 24 16:21:13.757596 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.757575 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="860566b7-7697-4722-b994-f25413231f03" containerName="cleanup" Jul 24 16:21:13.757661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.757601 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="860566b7-7697-4722-b994-f25413231f03" containerName="cleanup" Jul 24 16:21:13.757661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.757616 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="622850fb-3022-47fd-b57d-8e84978d4c0f" containerName="cleanup" Jul 24 16:21:13.757661 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.757623 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="622850fb-3022-47fd-b57d-8e84978d4c0f" containerName="cleanup" Jul 24 16:21:13.757804 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.757789 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="622850fb-3022-47fd-b57d-8e84978d4c0f" containerName="cleanup" Jul 24 16:21:13.757848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.757806 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="860566b7-7697-4722-b994-f25413231f03" containerName="cleanup" Jul 24 16:21:13.761343 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.761291 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.764529 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.764503 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-5d0582-openshift-default-dockercfg-z79dc\"" Jul 24 16:21:13.776588 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.776563 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8"] Jul 24 16:21:13.791734 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.791707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:13.909043 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909001 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-certs\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909043 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-data\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909286 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa191de6-2595-4db3-abe1-3c73170d31fd-istiod-ca-cert\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909286 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-envoy\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909286 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909133 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-token\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909286 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909191 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-podinfo\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909286 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-credential-socket\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909586 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-socket\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:13.909586 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:13.909441 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm26j\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-kube-api-access-rm26j\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.010721 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.010608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa191de6-2595-4db3-abe1-3c73170d31fd-istiod-ca-cert\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.010721 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.010674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-envoy\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.010721 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.010719 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-token\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011031 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.010749 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-podinfo\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011031 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.010779 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-credential-socket\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011031 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.010878 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-socket\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011031 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.010906 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm26j\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-kube-api-access-rm26j\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011031 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.011014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-certs\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011297 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.011041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-data\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011297 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.011266 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-socket\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011458 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.011391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-data\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011516 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.011485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-certs\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.011552 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-credential-socket\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.011573 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.011556 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa191de6-2595-4db3-abe1-3c73170d31fd-istiod-ca-cert\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.013920 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.013894 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-envoy\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.014038 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.014001 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-podinfo\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.018486 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.018457 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm26j\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-kube-api-access-rm26j\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.018610 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.018559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-token\") pod \"e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.032518 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.032492 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:14.072362 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.072333 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:14.221096 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.221072 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8"] Jul 24 16:21:14.223500 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:21:14.223448 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa191de6_2595_4db3_abe1_3c73170d31fd.slice/crio-b7c29ff3b3389ff86dfbb330af2edc6611a762ed30082c7546eaddd1e7f1315f WatchSource:0}: Error finding container b7c29ff3b3389ff86dfbb330af2edc6611a762ed30082c7546eaddd1e7f1315f: Status 404 returned error can't find the container with id b7c29ff3b3389ff86dfbb330af2edc6611a762ed30082c7546eaddd1e7f1315f Jul 24 16:21:14.226163 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.226126 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:14.226280 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.226211 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:14.226280 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.226262 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:14.814377 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.814333 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" event={"ID":"aa191de6-2595-4db3-abe1-3c73170d31fd","Type":"ContainerStarted","Data":"894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc"} Jul 24 16:21:14.814779 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.814444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" event={"ID":"aa191de6-2595-4db3-abe1-3c73170d31fd","Type":"ContainerStarted","Data":"b7c29ff3b3389ff86dfbb330af2edc6611a762ed30082c7546eaddd1e7f1315f"} Jul 24 16:21:14.836807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:14.836751 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" podStartSLOduration=1.836734176 podStartE2EDuration="1.836734176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:14.833749941 +0000 UTC m=+2698.256350995" watchObservedRunningTime="2026-07-24 16:21:14.836734176 +0000 UTC m=+2698.259335327" Jul 24 16:21:15.072667 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:15.072569 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:15.077641 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:15.077617 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:15.819454 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:15.819421 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:15.820430 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:15.820408 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:16.947637 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:16.947605 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:17.056767 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:17.056732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:17.456441 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:17.456411 2560 scope.go:117] "RemoveContainer" containerID="81b41d2e1b66788ac2c927f9cd5635ef10f66c3f1fa7835a8289866e31837a85" Jul 24 16:21:18.025750 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:18.025704 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:18.517144 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:18.517110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:18.618925 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:18.618894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:18.705158 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:18.705119 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:21.419474 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:21.419430 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:26.813756 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:26.813716 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:21:32.439777 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.439730 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:32.793793 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.793762 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d"] Jul 24 16:21:32.797237 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.797220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.800023 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.800000 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-07e89c-openshift-default-dockercfg-kmpdk\"" Jul 24 16:21:32.803787 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.803753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-socket\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.803890 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.803788 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-envoy\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.803890 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.803863 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-podinfo\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.804007 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.803927 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-credential-socket\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.804007 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.803962 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx5m2\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-kube-api-access-tx5m2\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.804109 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.804041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-certs\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.804109 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.804067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-data\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.804213 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.804113 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d35dd62d-9363-4637-ac0c-5afa422dbe31-istiod-ca-cert\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.804213 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.804158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-token\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.830225 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.830183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:32.833150 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.833123 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d"] Jul 24 16:21:32.905428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.904826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-token\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.904893 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-socket\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.904917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-envoy\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.905343 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-podinfo\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.905424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-credential-socket\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905861 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.905463 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx5m2\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-kube-api-access-tx5m2\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905861 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.905540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-certs\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905861 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.905563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-data\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.905861 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.905610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d35dd62d-9363-4637-ac0c-5afa422dbe31-istiod-ca-cert\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.907327 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.906405 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d35dd62d-9363-4637-ac0c-5afa422dbe31-istiod-ca-cert\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.907327 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.906705 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-socket\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.907327 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.907216 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-credential-socket\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.907580 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.907438 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-certs\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.907745 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.907723 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-data\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.909918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.909891 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-envoy\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.910504 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.910471 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-podinfo\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.914722 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.914687 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-token\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.915115 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.915093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx5m2\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-kube-api-access-tx5m2\") pod \"e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:32.989284 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:32.989242 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:33.108383 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.108256 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:33.268323 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:21:33.268266 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35dd62d_9363_4637_ac0c_5afa422dbe31.slice/crio-3a25f6f20c573f90f69293889d4f258d6690bcd9ca24e0b00dd7c02275845727 WatchSource:0}: Error finding container 3a25f6f20c573f90f69293889d4f258d6690bcd9ca24e0b00dd7c02275845727: Status 404 returned error can't find the container with id 3a25f6f20c573f90f69293889d4f258d6690bcd9ca24e0b00dd7c02275845727 Jul 24 16:21:33.269244 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.269218 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d"] Jul 24 16:21:33.270949 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.270917 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:33.271023 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.270983 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:33.271023 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.271015 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:33.923087 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.923054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" event={"ID":"d35dd62d-9363-4637-ac0c-5afa422dbe31","Type":"ContainerStarted","Data":"60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912"} Jul 24 16:21:33.923455 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.923104 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" event={"ID":"d35dd62d-9363-4637-ac0c-5afa422dbe31","Type":"ContainerStarted","Data":"3a25f6f20c573f90f69293889d4f258d6690bcd9ca24e0b00dd7c02275845727"} Jul 24 16:21:33.953855 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:33.953800 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" podStartSLOduration=1.953782238 podStartE2EDuration="1.953782238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:33.951664659 +0000 UTC m=+2717.374265708" watchObservedRunningTime="2026-07-24 16:21:33.953782238 +0000 UTC m=+2717.376383267" Jul 24 16:21:34.109476 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:34.109443 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:34.115027 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:34.114998 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:34.928135 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:34.928100 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:34.929434 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:34.929414 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:35.934530 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:35.934499 2560 generic.go:356] "Generic (PLEG): container finished" podID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerID="894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc" exitCode=0 Jul 24 16:21:35.935078 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:35.934573 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" event={"ID":"aa191de6-2595-4db3-abe1-3c73170d31fd","Type":"ContainerDied","Data":"894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc"} Jul 24 16:21:35.935159 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:35.935083 2560 scope.go:117] "RemoveContainer" containerID="894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc" Jul 24 16:21:35.936144 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:35.936113 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:35.936238 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:35.936195 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:35.936321 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:35.936241 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:35.975814 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:35.975770 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:36.065936 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:36.065896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:36.948765 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:36.948722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" event={"ID":"aa191de6-2595-4db3-abe1-3c73170d31fd","Type":"ContainerStarted","Data":"25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748"} Jul 24 16:21:37.072986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:37.072949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:37.073237 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:37.073212 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 16:21:37.073428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:37.073402 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 16:21:37.809116 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:37.809079 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:38.073075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:38.072983 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 16:21:38.073689 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:38.073657 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 16:21:38.274327 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:38.274277 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:38.376368 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:38.376245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:38.472918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:38.472879 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:39.073190 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:39.073142 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 16:21:39.073734 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:39.073217 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 16:21:40.073852 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:40.073808 2560 patch_prober.go:28] interesting pod/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 16:21:40.074394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:40.073883 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 16:21:40.445580 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:40.445490 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:21:41.076825 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:41.076794 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:41.077230 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:41.077073 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:41.077924 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:41.077903 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:21:44.991002 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:44.990959 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:21:54.049464 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:54.049426 2560 generic.go:356] "Generic (PLEG): container finished" podID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerID="60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912" exitCode=0 Jul 24 16:21:54.050032 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:54.049506 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" event={"ID":"d35dd62d-9363-4637-ac0c-5afa422dbe31","Type":"ContainerDied","Data":"60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912"} Jul 24 16:21:54.050032 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:54.050013 2560 scope.go:117] "RemoveContainer" containerID="60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912" Jul 24 16:21:54.051078 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:54.051051 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:54.051143 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:54.051120 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:54.051180 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:54.051152 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:21:55.057676 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:55.057639 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" event={"ID":"d35dd62d-9363-4637-ac0c-5afa422dbe31","Type":"ContainerStarted","Data":"8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a"} Jul 24 16:21:55.109089 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:55.109054 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:21:55.109389 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:55.109367 2560 patch_prober.go:28] interesting pod/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 16:21:55.109441 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:55.109416 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 16:21:56.109222 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:56.109185 2560 patch_prober.go:28] interesting pod/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 16:21:56.109645 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:56.109240 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 16:21:57.109241 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:57.109207 2560 patch_prober.go:28] interesting pod/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 16:21:57.109672 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:57.109260 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 16:21:58.109102 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:58.109057 2560 patch_prober.go:28] interesting pod/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 16:21:58.109356 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:58.109135 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 16:21:59.109018 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:59.108974 2560 patch_prober.go:28] interesting pod/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 16:21:59.109229 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:21:59.109051 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 16:22:00.112703 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:00.112665 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:22:00.113174 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:00.113064 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:22:00.113707 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:00.113687 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:22:08.743598 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:08.743557 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:13.196297 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:13.196255 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:22:28.603755 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:28.603716 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:22:40.253494 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.253395 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:40.548636 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.548595 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv"] Jul 24 16:22:40.552255 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.552233 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.554796 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.554774 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-ebf027-openshift-default-dockercfg-hxn8w\"" Jul 24 16:22:40.563957 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.563930 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv"] Jul 24 16:22:40.571775 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.571729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:40.639477 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639432 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35838ff2-01d2-431f-9c63-20b6ac8791bd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639674 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-certs\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639674 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639549 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-credential-socket\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639674 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639578 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-podinfo\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639674 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639622 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-envoy\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639674 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-token\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639738 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-socket\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-data\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.639978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.639816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7qg5\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-kube-api-access-w7qg5\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741194 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-envoy\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741422 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-token\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741422 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-socket\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741422 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741336 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-data\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741422 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7qg5\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-kube-api-access-w7qg5\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741422 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35838ff2-01d2-431f-9c63-20b6ac8791bd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741694 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-certs\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741694 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-credential-socket\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741694 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741549 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-podinfo\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741852 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741733 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-socket\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741852 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741784 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-data\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741965 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741922 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-credential-socket\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.741965 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.741925 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-certs\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.742477 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.742450 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35838ff2-01d2-431f-9c63-20b6ac8791bd-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.743764 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.743746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-envoy\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.744406 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.744388 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-podinfo\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.758066 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.758037 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-token\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.758183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.758045 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7qg5\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-kube-api-access-w7qg5\") pod \"e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.864336 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.864213 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:40.875148 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:40.875116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:41.016845 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.016815 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv"] Jul 24 16:22:41.019188 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:22:41.019160 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35838ff2_01d2_431f_9c63_20b6ac8791bd.slice/crio-89e01db971293b571f199e4af64486ed24810d1c4f6ce2cc8693f17168f23c2a WatchSource:0}: Error finding container 89e01db971293b571f199e4af64486ed24810d1c4f6ce2cc8693f17168f23c2a: Status 404 returned error can't find the container with id 89e01db971293b571f199e4af64486ed24810d1c4f6ce2cc8693f17168f23c2a Jul 24 16:22:41.021452 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.021420 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:22:41.021521 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.021489 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:22:41.021559 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.021519 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:22:41.317086 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.317048 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" event={"ID":"35838ff2-01d2-431f-9c63-20b6ac8791bd","Type":"ContainerStarted","Data":"766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3"} Jul 24 16:22:41.317539 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.317118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" event={"ID":"35838ff2-01d2-431f-9c63-20b6ac8791bd","Type":"ContainerStarted","Data":"89e01db971293b571f199e4af64486ed24810d1c4f6ce2cc8693f17168f23c2a"} Jul 24 16:22:41.347588 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.347536 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" podStartSLOduration=1.347518693 podStartE2EDuration="1.347518693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:22:41.343817523 +0000 UTC m=+2784.766418574" watchObservedRunningTime="2026-07-24 16:22:41.347518693 +0000 UTC m=+2784.770119722" Jul 24 16:22:41.864837 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:41.864800 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:42.869756 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:42.869726 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:43.327633 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:43.327597 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:43.328594 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:43.328570 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:22:43.426566 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:43.426533 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:43.524755 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:43.524702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:46.394378 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:46.394344 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:46.654230 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:46.654128 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:46.841261 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:46.841227 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:46.930510 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:46.930420 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:47.871710 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:47.871672 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:22:51.095963 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:51.095924 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:22:59.335075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.334829 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 16:22:59.335075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.334919 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 16:22:59.420338 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.420264 2560 generic.go:356] "Generic (PLEG): container finished" podID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerID="766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3" exitCode=0 Jul 24 16:22:59.420519 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.420354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" event={"ID":"35838ff2-01d2-431f-9c63-20b6ac8791bd","Type":"ContainerDied","Data":"766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3"} Jul 24 16:22:59.420895 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.420874 2560 scope.go:117] "RemoveContainer" containerID="766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3" Jul 24 16:22:59.422291 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.422259 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:22:59.422394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.422350 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:22:59.422394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:22:59.422382 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 16:23:00.427687 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:00.427653 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" event={"ID":"35838ff2-01d2-431f-9c63-20b6ac8791bd","Type":"ContainerStarted","Data":"f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0"} Jul 24 16:23:00.865406 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:00.865363 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:23:00.865773 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:00.865742 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 16:23:00.866015 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:00.865988 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 16:23:01.865799 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:01.865745 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 16:23:01.866355 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:01.865844 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 16:23:02.864870 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:02.864830 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 16:23:02.865207 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:02.865177 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 16:23:03.865798 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:03.865754 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 16:23:03.866341 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:03.865822 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 16:23:04.869026 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:04.868991 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:23:04.869441 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:04.869332 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:23:04.870212 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:04.870191 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:23:14.248285 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.248246 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx"] Jul 24 16:23:14.251039 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.251020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" Jul 24 16:23:14.253791 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.253769 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:23:14.254219 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.254199 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:23:14.259891 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.259866 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx"] Jul 24 16:23:14.311972 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.311933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7215cb-bdec-477d-b91e-877ee396b35d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx\" (UID: \"6d7215cb-bdec-477d-b91e-877ee396b35d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" Jul 24 16:23:14.413202 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.413164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7215cb-bdec-477d-b91e-877ee396b35d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx\" (UID: \"6d7215cb-bdec-477d-b91e-877ee396b35d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" Jul 24 16:23:14.413892 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.413866 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7215cb-bdec-477d-b91e-877ee396b35d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx\" (UID: \"6d7215cb-bdec-477d-b91e-877ee396b35d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" Jul 24 16:23:14.563806 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.563766 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" Jul 24 16:23:14.706944 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:14.706918 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx"] Jul 24 16:23:14.709540 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:23:14.709508 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7215cb_bdec_477d_b91e_877ee396b35d.slice/crio-ba06549073230a01c2befec1498c6b20301070619a8513f34b8ee5574f0aa040 WatchSource:0}: Error finding container ba06549073230a01c2befec1498c6b20301070619a8513f34b8ee5574f0aa040: Status 404 returned error can't find the container with id ba06549073230a01c2befec1498c6b20301070619a8513f34b8ee5574f0aa040 Jul 24 16:23:15.510960 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:15.510921 2560 generic.go:356] "Generic (PLEG): container finished" podID="6d7215cb-bdec-477d-b91e-877ee396b35d" containerID="5a0c80e6838f5b265cabf725bc1d44507a4c917ec822e47dcfbc637c1de7056d" exitCode=0 Jul 24 16:23:15.511364 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:15.511003 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" event={"ID":"6d7215cb-bdec-477d-b91e-877ee396b35d","Type":"ContainerDied","Data":"5a0c80e6838f5b265cabf725bc1d44507a4c917ec822e47dcfbc637c1de7056d"} Jul 24 16:23:15.511364 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:15.511060 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" event={"ID":"6d7215cb-bdec-477d-b91e-877ee396b35d","Type":"ContainerStarted","Data":"ba06549073230a01c2befec1498c6b20301070619a8513f34b8ee5574f0aa040"} Jul 24 16:23:16.657049 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:16.657008 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" Jul 24 16:23:16.736537 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:16.736497 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7215cb-bdec-477d-b91e-877ee396b35d-maas-api-service-ca\") pod \"6d7215cb-bdec-477d-b91e-877ee396b35d\" (UID: \"6d7215cb-bdec-477d-b91e-877ee396b35d\") " Jul 24 16:23:16.736869 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:16.736847 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d7215cb-bdec-477d-b91e-877ee396b35d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d7215cb-bdec-477d-b91e-877ee396b35d" (UID: "6d7215cb-bdec-477d-b91e-877ee396b35d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:23:16.838025 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:16.837941 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7215cb-bdec-477d-b91e-877ee396b35d-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:17.521983 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:17.521945 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" Jul 24 16:23:17.522182 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:17.521982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx" event={"ID":"6d7215cb-bdec-477d-b91e-877ee396b35d","Type":"ContainerDied","Data":"ba06549073230a01c2befec1498c6b20301070619a8513f34b8ee5574f0aa040"} Jul 24 16:23:17.522182 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:17.522027 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba06549073230a01c2befec1498c6b20301070619a8513f34b8ee5574f0aa040" Jul 24 16:23:26.914241 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:26.914201 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:27.025602 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:27.025571 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:27.124905 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:27.124872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:29.325992 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:29.325947 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx"] Jul 24 16:23:29.328719 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:29.328692 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:29.330229 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:29.330206 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebf027-8rqbx"] Jul 24 16:23:29.422856 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:29.422823 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:29.721988 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:29.721900 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:31.141557 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:31.141524 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d7215cb-bdec-477d-b91e-877ee396b35d" path="/var/lib/kubelet/pods/6d7215cb-bdec-477d-b91e-877ee396b35d/volumes" Jul 24 16:23:32.208755 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:32.208718 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:23:35.873057 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:35.873018 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:35.974816 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:35.974766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:36.132271 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:36.132189 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv"] Jul 24 16:23:36.132435 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:36.132417 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" containerID="cri-o://f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0" gracePeriod=30 Jul 24 16:23:36.151042 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:36.151010 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:36.384450 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:36.384355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:39.981337 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.981288 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25"] Jul 24 16:23:39.981733 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.981723 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7215cb-bdec-477d-b91e-877ee396b35d" containerName="revoke-keys" Jul 24 16:23:39.981790 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.981735 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7215cb-bdec-477d-b91e-877ee396b35d" containerName="revoke-keys" Jul 24 16:23:39.981833 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.981800 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7215cb-bdec-477d-b91e-877ee396b35d" containerName="revoke-keys" Jul 24 16:23:39.984880 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.984863 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" Jul 24 16:23:39.987229 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.987209 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\"" Jul 24 16:23:39.987347 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.987209 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 16:23:39.991728 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:39.991703 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25"] Jul 24 16:23:40.082161 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:40.082124 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5684f10-86e2-4734-b566-185ad6634dad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25\" (UID: \"e5684f10-86e2-4734-b566-185ad6634dad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" Jul 24 16:23:40.182805 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:40.182758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5684f10-86e2-4734-b566-185ad6634dad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25\" (UID: \"e5684f10-86e2-4734-b566-185ad6634dad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" Jul 24 16:23:40.183441 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:40.183420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5684f10-86e2-4734-b566-185ad6634dad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25\" (UID: \"e5684f10-86e2-4734-b566-185ad6634dad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" Jul 24 16:23:40.297142 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:40.297108 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" Jul 24 16:23:40.445637 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:40.445613 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25"] Jul 24 16:23:40.447787 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:23:40.447759 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5684f10_86e2_4734_b566_185ad6634dad.slice/crio-5d51a9baf4e14e9bc765323327f7532c916d295bdb8900fe71961e010fe70b0c WatchSource:0}: Error finding container 5d51a9baf4e14e9bc765323327f7532c916d295bdb8900fe71961e010fe70b0c: Status 404 returned error can't find the container with id 5d51a9baf4e14e9bc765323327f7532c916d295bdb8900fe71961e010fe70b0c Jul 24 16:23:40.647952 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:40.647860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" event={"ID":"e5684f10-86e2-4734-b566-185ad6634dad","Type":"ContainerStarted","Data":"5d51a9baf4e14e9bc765323327f7532c916d295bdb8900fe71961e010fe70b0c"} Jul 24 16:23:41.392633 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.392607 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:23:41.498812 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.498724 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-certs\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.498812 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.498763 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-data\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.498812 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.498787 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-token\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.499047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.498869 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-credential-socket\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.499047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.498891 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-envoy\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.499047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.498937 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-podinfo\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.499047 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499009 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-socket\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.499257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499046 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7qg5\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-kube-api-access-w7qg5\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.499257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499074 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35838ff2-01d2-431f-9c63-20b6ac8791bd-istiod-ca-cert\") pod \"35838ff2-01d2-431f-9c63-20b6ac8791bd\" (UID: \"35838ff2-01d2-431f-9c63-20b6ac8791bd\") " Jul 24 16:23:41.499257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499087 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:23:41.499257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499209 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:23:41.499498 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499361 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:23:41.499575 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499547 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35838ff2-01d2-431f-9c63-20b6ac8791bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:23:41.499632 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499594 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.499632 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499620 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.499736 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499637 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.499967 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.499870 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:23:41.501260 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.501232 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:23:41.501469 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.501449 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:23:41.501550 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.501450 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:23:41.501550 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.501538 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-kube-api-access-w7qg5" (OuterVolumeSpecName: "kube-api-access-w7qg5") pod "35838ff2-01d2-431f-9c63-20b6ac8791bd" (UID: "35838ff2-01d2-431f-9c63-20b6ac8791bd"). InnerVolumeSpecName "kube-api-access-w7qg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:23:41.600229 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.600185 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7qg5\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-kube-api-access-w7qg5\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.600229 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.600221 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35838ff2-01d2-431f-9c63-20b6ac8791bd-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.600229 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.600235 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.600509 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.600245 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.600509 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.600256 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.600509 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.600265 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35838ff2-01d2-431f-9c63-20b6ac8791bd-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:41.658620 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.658385 2560 generic.go:356] "Generic (PLEG): container finished" podID="e5684f10-86e2-4734-b566-185ad6634dad" containerID="26e44df24eab482ec8858c1e692c3246ef09c767cb7de7ad6406e239d3baf232" exitCode=0 Jul 24 16:23:41.658620 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.658443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" event={"ID":"e5684f10-86e2-4734-b566-185ad6634dad","Type":"ContainerDied","Data":"26e44df24eab482ec8858c1e692c3246ef09c767cb7de7ad6406e239d3baf232"} Jul 24 16:23:41.661053 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.661018 2560 generic.go:356] "Generic (PLEG): container finished" podID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerID="f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0" exitCode=0 Jul 24 16:23:41.661199 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.661094 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" Jul 24 16:23:41.661199 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.661089 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" event={"ID":"35838ff2-01d2-431f-9c63-20b6ac8791bd","Type":"ContainerDied","Data":"f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0"} Jul 24 16:23:41.661333 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.661214 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv" event={"ID":"35838ff2-01d2-431f-9c63-20b6ac8791bd","Type":"ContainerDied","Data":"89e01db971293b571f199e4af64486ed24810d1c4f6ce2cc8693f17168f23c2a"} Jul 24 16:23:41.661333 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.661242 2560 scope.go:117] "RemoveContainer" containerID="f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0" Jul 24 16:23:41.672698 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.672676 2560 scope.go:117] "RemoveContainer" containerID="766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3" Jul 24 16:23:41.681532 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.681509 2560 scope.go:117] "RemoveContainer" containerID="f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0" Jul 24 16:23:41.681846 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:23:41.681822 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0\": container with ID starting with f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0 not found: ID does not exist" containerID="f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0" Jul 24 16:23:41.681918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.681855 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0"} err="failed to get container status \"f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0\": rpc error: code = NotFound desc = could not find container \"f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0\": container with ID starting with f3f44a9039d3c9754205a429bcfdea2115f28757f40968b7f0cec9a469268ad0 not found: ID does not exist" Jul 24 16:23:41.681918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.681883 2560 scope.go:117] "RemoveContainer" containerID="766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3" Jul 24 16:23:41.682121 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:23:41.682100 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3\": container with ID starting with 766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3 not found: ID does not exist" containerID="766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3" Jul 24 16:23:41.682159 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.682128 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3"} err="failed to get container status \"766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3\": rpc error: code = NotFound desc = could not find container \"766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3\": container with ID starting with 766e35eab091819152cae275bae5069b501b5451d06a1cbd5b5da44b09b8abb3 not found: ID does not exist" Jul 24 16:23:41.704169 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.704135 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv"] Jul 24 16:23:41.710930 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:41.710905 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ebf027-openshift-default-86d4dd87d4-vcdgv"] Jul 24 16:23:42.805436 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:42.805413 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" Jul 24 16:23:42.912353 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:42.912294 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5684f10-86e2-4734-b566-185ad6634dad-maas-api-service-ca\") pod \"e5684f10-86e2-4734-b566-185ad6634dad\" (UID: \"e5684f10-86e2-4734-b566-185ad6634dad\") " Jul 24 16:23:42.912659 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:42.912638 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5684f10-86e2-4734-b566-185ad6634dad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5684f10-86e2-4734-b566-185ad6634dad" (UID: "e5684f10-86e2-4734-b566-185ad6634dad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:23:43.013248 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:43.013213 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5684f10-86e2-4734-b566-185ad6634dad-maas-api-service-ca\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:23:43.140953 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:43.140906 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" path="/var/lib/kubelet/pods/35838ff2-01d2-431f-9c63-20b6ac8791bd/volumes" Jul 24 16:23:43.675611 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:43.675578 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" Jul 24 16:23:43.675611 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:43.675596 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25" event={"ID":"e5684f10-86e2-4734-b566-185ad6634dad","Type":"ContainerDied","Data":"5d51a9baf4e14e9bc765323327f7532c916d295bdb8900fe71961e010fe70b0c"} Jul 24 16:23:43.675850 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:43.675652 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d51a9baf4e14e9bc765323327f7532c916d295bdb8900fe71961e010fe70b0c" Jul 24 16:23:56.826040 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:56.825998 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:57.715943 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:57.715909 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:23:59.750994 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:59.750955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:59.850812 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:59.850768 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:23:59.917454 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:23:59.917420 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:00.128639 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:00.128609 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25"] Jul 24 16:24:00.131900 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:00.131871 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-5d0582-zgn25"] Jul 24 16:24:00.132055 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:00.131990 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:00.761356 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:00.761285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:01.141549 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:01.141470 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5684f10-86e2-4734-b566-185ad6634dad" path="/var/lib/kubelet/pods/e5684f10-86e2-4734-b566-185ad6634dad/volumes" Jul 24 16:24:05.653357 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:05.653323 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:09.701453 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:09.701366 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:24:24.120869 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:24.120830 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:24.217565 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:24.217527 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:42.486722 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:42.486680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:44.224970 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:44.224939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:49.831734 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:49.831701 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:55.930517 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:55.930476 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:56.049890 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:56.049848 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:56.256807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:56.256714 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8"] Jul 24 16:24:56.257254 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:56.257212 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" containerID="cri-o://25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748" gracePeriod=30 Jul 24 16:24:56.283893 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:56.283852 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:24:56.572683 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:24:56.572645 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:01.528413 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.528339 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:25:01.702811 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.702774 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-podinfo\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.702969 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.702849 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-certs\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.702969 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.702880 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-token\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.702969 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.702925 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-data\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.702969 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.702960 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-socket\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.703190 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703032 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm26j\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-kube-api-access-rm26j\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.703190 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703074 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-envoy\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.703190 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703096 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-credential-socket\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.703190 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703121 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:01.703190 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703165 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa191de6-2595-4db3-abe1-3c73170d31fd-istiod-ca-cert\") pod \"aa191de6-2595-4db3-abe1-3c73170d31fd\" (UID: \"aa191de6-2595-4db3-abe1-3c73170d31fd\") " Jul 24 16:25:01.703496 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703419 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:01.703496 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703452 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:01.703932 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703845 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.703932 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703872 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.703932 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703888 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.703932 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703897 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa191de6-2595-4db3-abe1-3c73170d31fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:25:01.704240 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.703951 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:01.705470 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.705444 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:25:01.705470 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.705457 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:25:01.705616 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.705470 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-kube-api-access-rm26j" (OuterVolumeSpecName: "kube-api-access-rm26j") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "kube-api-access-rm26j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:25:01.705616 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.705601 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa191de6-2595-4db3-abe1-3c73170d31fd" (UID: "aa191de6-2595-4db3-abe1-3c73170d31fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:01.805360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.805241 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa191de6-2595-4db3-abe1-3c73170d31fd-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.805360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.805274 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.805360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.805285 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.805360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.805330 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.805360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.805344 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm26j\" (UniqueName: \"kubernetes.io/projected/aa191de6-2595-4db3-abe1-3c73170d31fd-kube-api-access-rm26j\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:01.805360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:01.805356 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa191de6-2595-4db3-abe1-3c73170d31fd-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:02.090166 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.090063 2560 generic.go:356] "Generic (PLEG): container finished" podID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerID="25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748" exitCode=0 Jul 24 16:25:02.090166 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.090125 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" Jul 24 16:25:02.090166 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.090157 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" event={"ID":"aa191de6-2595-4db3-abe1-3c73170d31fd","Type":"ContainerDied","Data":"25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748"} Jul 24 16:25:02.090461 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.090262 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8" event={"ID":"aa191de6-2595-4db3-abe1-3c73170d31fd","Type":"ContainerDied","Data":"b7c29ff3b3389ff86dfbb330af2edc6611a762ed30082c7546eaddd1e7f1315f"} Jul 24 16:25:02.090461 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.090297 2560 scope.go:117] "RemoveContainer" containerID="25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748" Jul 24 16:25:02.108264 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.108239 2560 scope.go:117] "RemoveContainer" containerID="894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc" Jul 24 16:25:02.117360 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.117277 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8"] Jul 24 16:25:02.117734 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.117707 2560 scope.go:117] "RemoveContainer" containerID="25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748" Jul 24 16:25:02.118050 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:25:02.118030 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748\": container with ID starting with 25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748 not found: ID does not exist" containerID="25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748" Jul 24 16:25:02.118108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.118059 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748"} err="failed to get container status \"25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748\": rpc error: code = NotFound desc = could not find container \"25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748\": container with ID starting with 25af68d15054693a379c211ed7e8152332398df6e1bec3c074a36ce63915c748 not found: ID does not exist" Jul 24 16:25:02.118108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.118076 2560 scope.go:117] "RemoveContainer" containerID="894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc" Jul 24 16:25:02.118336 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:25:02.118315 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc\": container with ID starting with 894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc not found: ID does not exist" containerID="894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc" Jul 24 16:25:02.118402 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.118347 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc"} err="failed to get container status \"894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc\": rpc error: code = NotFound desc = could not find container \"894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc\": container with ID starting with 894a1604a0f28c90dbadb39bea0eca4150778130f6fa9afbc3296a0fcf09b0cc not found: ID does not exist" Jul 24 16:25:02.121923 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:02.121899 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d0582-openshift-default-5f875678f8-ld8x8"] Jul 24 16:25:03.141024 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:03.140987 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" path="/var/lib/kubelet/pods/aa191de6-2595-4db3-abe1-3c73170d31fd/volumes" Jul 24 16:25:06.071445 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:06.071388 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:06.746739 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:06.746703 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:06.824352 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:06.824269 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:06.885070 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:06.885033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:10.495744 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:10.495708 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:25:11.769056 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:11.769021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:12.197247 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:12.197148 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:18.506458 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:18.506427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:18.610108 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:18.610080 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:18.776065 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:18.776025 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d"] Jul 24 16:25:18.776259 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:18.776236 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" containerID="cri-o://8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a" gracePeriod=30 Jul 24 16:25:18.803176 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:18.803130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:19.031420 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:19.031289 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:24.036099 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.036074 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:25:24.140170 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140044 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-data\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140170 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140134 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-envoy\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140176 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-credential-socket\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140207 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx5m2\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-kube-api-access-tx5m2\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140263 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-podinfo\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140335 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d35dd62d-9363-4637-ac0c-5afa422dbe31-istiod-ca-cert\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140386 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-socket\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140436 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-certs\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140473 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140465 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-token\") pod \"d35dd62d-9363-4637-ac0c-5afa422dbe31\" (UID: \"d35dd62d-9363-4637-ac0c-5afa422dbe31\") " Jul 24 16:25:24.140827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140509 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:24.140827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140696 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:24.140827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140712 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:24.140999 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140890 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.140999 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140912 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.140999 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140919 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-data" (OuterVolumeSpecName: "istio-data") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:24.140999 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140926 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.141141 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.140997 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d35dd62d-9363-4637-ac0c-5afa422dbe31-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:25:24.142707 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.142675 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-kube-api-access-tx5m2" (OuterVolumeSpecName: "kube-api-access-tx5m2") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "kube-api-access-tx5m2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:25:24.142928 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.142913 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-token" (OuterVolumeSpecName: "istio-token") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:25:24.143093 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.143067 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:24.143093 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.143074 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d35dd62d-9363-4637-ac0c-5afa422dbe31" (UID: "d35dd62d-9363-4637-ac0c-5afa422dbe31"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:25:24.209199 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.209159 2560 generic.go:356] "Generic (PLEG): container finished" podID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerID="8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a" exitCode=0 Jul 24 16:25:24.209394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.209214 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" Jul 24 16:25:24.209394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.209246 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" event={"ID":"d35dd62d-9363-4637-ac0c-5afa422dbe31","Type":"ContainerDied","Data":"8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a"} Jul 24 16:25:24.209394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.209318 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d" event={"ID":"d35dd62d-9363-4637-ac0c-5afa422dbe31","Type":"ContainerDied","Data":"3a25f6f20c573f90f69293889d4f258d6690bcd9ca24e0b00dd7c02275845727"} Jul 24 16:25:24.209394 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.209331 2560 scope.go:117] "RemoveContainer" containerID="8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a" Jul 24 16:25:24.221790 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.221751 2560 scope.go:117] "RemoveContainer" containerID="60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912" Jul 24 16:25:24.230766 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.230745 2560 scope.go:117] "RemoveContainer" containerID="8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a" Jul 24 16:25:24.231083 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:25:24.231060 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a\": container with ID starting with 8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a not found: ID does not exist" containerID="8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a" Jul 24 16:25:24.231139 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.231096 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a"} err="failed to get container status \"8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a\": rpc error: code = NotFound desc = could not find container \"8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a\": container with ID starting with 8e56e7516078a8eb4f7dd2894d1563cb093e499e2410f589f75a4b749ae23f0a not found: ID does not exist" Jul 24 16:25:24.231139 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.231116 2560 scope.go:117] "RemoveContainer" containerID="60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912" Jul 24 16:25:24.231389 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:25:24.231370 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912\": container with ID starting with 60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912 not found: ID does not exist" containerID="60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912" Jul 24 16:25:24.231456 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.231397 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912"} err="failed to get container status \"60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912\": rpc error: code = NotFound desc = could not find container \"60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912\": container with ID starting with 60948008a372a0a7ba487b718f7827c8c9b8566f76fb9ef5cb9e80b0b11ea912 not found: ID does not exist" Jul 24 16:25:24.240510 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.240481 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d"] Jul 24 16:25:24.241744 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.241725 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.241807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.241747 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.241807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.241755 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.241807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.241764 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx5m2\" (UniqueName: \"kubernetes.io/projected/d35dd62d-9363-4637-ac0c-5afa422dbe31-kube-api-access-tx5m2\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.241807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.241772 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d35dd62d-9363-4637-ac0c-5afa422dbe31-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.241807 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.241783 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d35dd62d-9363-4637-ac0c-5afa422dbe31-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:24.253614 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:24.253575 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-07e89c-openshift-default-55f5cb9544-9tq6d"] Jul 24 16:25:25.147931 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:25.147895 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" path="/var/lib/kubelet/pods/d35dd62d-9363-4637-ac0c-5afa422dbe31/volumes" Jul 24 16:25:31.466442 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:31.466409 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b"] Jul 24 16:25:31.470757 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:31.470726 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-fafc28-29748495-vhw6b"] Jul 24 16:25:31.474766 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:31.474734 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:31.553101 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:31.553072 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:31.652066 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:31.652014 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:33.140425 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:33.140388 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="860566b7-7697-4722-b994-f25413231f03" path="/var/lib/kubelet/pods/860566b7-7697-4722-b994-f25413231f03/volumes" Jul 24 16:25:34.318772 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:34.318730 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:34.803178 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:34.803138 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:34.905499 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:34.905463 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:25:41.374350 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:41.374218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:41.474500 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:41.474465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:41.656185 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:41.656029 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296"] Jul 24 16:25:41.656377 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:41.656332 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" containerID="cri-o://7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449" gracePeriod=30 Jul 24 16:25:41.679183 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:41.679149 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:41.926918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:41.926814 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:46.918986 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.918960 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:25:46.960256 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960178 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-data\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.960437 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960263 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-envoy\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.960437 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960286 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-certs\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.960437 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960335 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882a76ba-6c28-4231-8080-6dab2bee7399-istio-podinfo\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.960437 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960355 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882a76ba-6c28-4231-8080-6dab2bee7399-istiod-ca-cert\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.960437 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960375 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-credential-socket\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.960437 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960428 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-istio-token\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960463 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8bvm\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-kube-api-access-w8bvm\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960496 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-socket\") pod \"882a76ba-6c28-4231-8080-6dab2bee7399\" (UID: \"882a76ba-6c28-4231-8080-6dab2bee7399\") " Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960660 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960868 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960885 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/882a76ba-6c28-4231-8080-6dab2bee7399-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960952 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960975 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:46.961016 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.960998 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:46.961663 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.961403 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-data" (OuterVolumeSpecName: "istio-data") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:46.962940 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.962917 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/882a76ba-6c28-4231-8080-6dab2bee7399-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:25:46.963087 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.963064 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-kube-api-access-w8bvm" (OuterVolumeSpecName: "kube-api-access-w8bvm") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "kube-api-access-w8bvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:25:46.963357 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.963339 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:25:46.963504 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:46.963483 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-istio-token" (OuterVolumeSpecName: "istio-token") pod "882a76ba-6c28-4231-8080-6dab2bee7399" (UID: "882a76ba-6c28-4231-8080-6dab2bee7399"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:25:47.061531 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.061488 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:47.061531 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.061523 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8bvm\" (UniqueName: \"kubernetes.io/projected/882a76ba-6c28-4231-8080-6dab2bee7399-kube-api-access-w8bvm\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:47.061531 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.061536 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:47.061785 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.061546 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:47.061785 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.061558 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882a76ba-6c28-4231-8080-6dab2bee7399-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:47.061785 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.061571 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882a76ba-6c28-4231-8080-6dab2bee7399-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:47.061785 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.061582 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882a76ba-6c28-4231-8080-6dab2bee7399-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:25:47.328741 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.328701 2560 generic.go:356] "Generic (PLEG): container finished" podID="882a76ba-6c28-4231-8080-6dab2bee7399" containerID="7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449" exitCode=0 Jul 24 16:25:47.328923 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.328752 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" Jul 24 16:25:47.328923 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.328770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" event={"ID":"882a76ba-6c28-4231-8080-6dab2bee7399","Type":"ContainerDied","Data":"7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449"} Jul 24 16:25:47.328923 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.328817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296" event={"ID":"882a76ba-6c28-4231-8080-6dab2bee7399","Type":"ContainerDied","Data":"faafd7ebb849ceb54cfa301d0c1ae7f0e81558d78448f514c1a46cd12ccd342e"} Jul 24 16:25:47.328923 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.328840 2560 scope.go:117] "RemoveContainer" containerID="7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449" Jul 24 16:25:47.337678 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.337661 2560 scope.go:117] "RemoveContainer" containerID="e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd" Jul 24 16:25:47.346270 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.346248 2560 scope.go:117] "RemoveContainer" containerID="7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449" Jul 24 16:25:47.346557 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:25:47.346533 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449\": container with ID starting with 7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449 not found: ID does not exist" containerID="7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449" Jul 24 16:25:47.346625 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.346569 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449"} err="failed to get container status \"7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449\": rpc error: code = NotFound desc = could not find container \"7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449\": container with ID starting with 7f9e341089f60e4175a4bde54a17c99e1bd1e8af903752bbd21449bbbdc94449 not found: ID does not exist" Jul 24 16:25:47.346625 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.346596 2560 scope.go:117] "RemoveContainer" containerID="e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd" Jul 24 16:25:47.346820 ip-10-0-132-149 kubenswrapper[2560]: E0724 16:25:47.346801 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd\": container with ID starting with e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd not found: ID does not exist" containerID="e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd" Jul 24 16:25:47.346864 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.346825 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd"} err="failed to get container status \"e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd\": rpc error: code = NotFound desc = could not find container \"e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd\": container with ID starting with e5ecff0d83b9f075ec8d3094f90ea1ad49ac35ad491080f5d5c3f60ec74dfcdd not found: ID does not exist" Jul 24 16:25:47.360024 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.359996 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296"] Jul 24 16:25:47.373517 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:47.373483 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-fafc28-openshift-default-865967cd4-45296"] Jul 24 16:25:49.141614 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:49.141558 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" path="/var/lib/kubelet/pods/882a76ba-6c28-4231-8080-6dab2bee7399/volumes" Jul 24 16:25:52.812771 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:52.812727 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:52.913668 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:52.913624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:53.024505 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:53.024469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:54.988069 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:54.988029 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8psn9"] Jul 24 16:25:57.240974 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:57.240937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:25:57.657027 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:25:57.656987 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:26:04.867519 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:04.867475 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:26:05.104058 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:05.104015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:26:05.267287 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:05.267257 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4"] Jul 24 16:26:05.267547 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:05.267519 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" containerID="cri-o://883011595cc9a82fb347b0b2f5d1b0083bccbdc373363ac24c6dd8b4b941801d" gracePeriod=30 Jul 24 16:26:05.317579 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:05.317537 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:26:05.536527 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:05.536394 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:26:10.453139 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.453099 2560 generic.go:356] "Generic (PLEG): container finished" podID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerID="883011595cc9a82fb347b0b2f5d1b0083bccbdc373363ac24c6dd8b4b941801d" exitCode=0 Jul 24 16:26:10.453589 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.453172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" event={"ID":"df6a470b-c088-4ae5-9597-7eba021b8cb5","Type":"ContainerDied","Data":"883011595cc9a82fb347b0b2f5d1b0083bccbdc373363ac24c6dd8b4b941801d"} Jul 24 16:26:10.453589 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.453250 2560 scope.go:117] "RemoveContainer" containerID="b5028a4d9f61390d4e22b4e08b97ad2a99f905d0d7f1c0e6d3a9980a1d56e35a" Jul 24 16:26:10.523918 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.523892 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:26:10.600685 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600593 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-token\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600685 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600640 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmvh6\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-kube-api-access-bmvh6\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600685 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600676 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-envoy\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600698 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-credential-socket\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600758 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-certs\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600782 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-podinfo\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600823 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-data\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600875 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6a470b-c088-4ae5-9597-7eba021b8cb5-istiod-ca-cert\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.600978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.600915 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-socket\") pod \"df6a470b-c088-4ae5-9597-7eba021b8cb5\" (UID: \"df6a470b-c088-4ae5-9597-7eba021b8cb5\") " Jul 24 16:26:10.601283 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.601028 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:26:10.601283 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.601070 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:26:10.601283 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.601247 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:26:10.601440 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.601336 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-credential-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.601440 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.601360 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-certs\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.601630 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.601601 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df6a470b-c088-4ae5-9597-7eba021b8cb5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 16:26:10.602087 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.602064 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-data" (OuterVolumeSpecName: "istio-data") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:26:10.603135 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.603114 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-token" (OuterVolumeSpecName: "istio-token") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:26:10.603135 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.603128 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 16:26:10.603624 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.603594 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 16:26:10.603624 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.603599 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-kube-api-access-bmvh6" (OuterVolumeSpecName: "kube-api-access-bmvh6") pod "df6a470b-c088-4ae5-9597-7eba021b8cb5" (UID: "df6a470b-c088-4ae5-9597-7eba021b8cb5"). InnerVolumeSpecName "kube-api-access-bmvh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 16:26:10.702037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.701986 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df6a470b-c088-4ae5-9597-7eba021b8cb5-istiod-ca-cert\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.702037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.702030 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-workload-socket\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.702037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.702040 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-token\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.702037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.702049 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmvh6\" (UniqueName: \"kubernetes.io/projected/df6a470b-c088-4ae5-9597-7eba021b8cb5-kube-api-access-bmvh6\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.702037 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.702059 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-envoy\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.702379 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.702067 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-podinfo\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:10.702379 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:10.702074 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df6a470b-c088-4ae5-9597-7eba021b8cb5-istio-data\") on node \"ip-10-0-132-149.ec2.internal\" DevicePath \"\"" Jul 24 16:26:11.459734 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:11.459653 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" Jul 24 16:26:11.459734 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:11.459698 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4" event={"ID":"df6a470b-c088-4ae5-9597-7eba021b8cb5","Type":"ContainerDied","Data":"371951e23cabcd88a05859efa83b02068697828aa946514a054eb5469e2dae3f"} Jul 24 16:26:11.460231 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:11.459754 2560 scope.go:117] "RemoveContainer" containerID="883011595cc9a82fb347b0b2f5d1b0083bccbdc373363ac24c6dd8b4b941801d" Jul 24 16:26:11.479075 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:11.479046 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4"] Jul 24 16:26:11.484915 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:11.484882 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c6c91e-openshift-default-6649b646d-mqfp4"] Jul 24 16:26:12.946244 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:12.946215 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-n54rh_39ac11c7-c954-4499-8b76-fac6954bc85d/copy-manifests/0.log" Jul 24 16:26:12.970613 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:12.970578 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-n54rh_39ac11c7-c954-4499-8b76-fac6954bc85d/manager/0.log" Jul 24 16:26:13.141455 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:13.141423 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" path="/var/lib/kubelet/pods/df6a470b-c088-4ae5-9597-7eba021b8cb5/volumes" Jul 24 16:26:13.595059 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:13.595025 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748480-mshkn_1cc85fd8-9564-4abd-b74b-59c3d496e102/cleanup/0.log" Jul 24 16:26:13.708888 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:13.708856 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748495-m77lh_622850fb-3022-47fd-b57d-8e84978d4c0f/cleanup/0.log" Jul 24 16:26:15.898799 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:15.898770 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8psn9_98c8b36a-1f80-4e3a-925f-a372b30ff830/limitador/0.log" Jul 24 16:26:16.398474 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:16.398433 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp_df4dfeac-3176-418d-a129-3438754d8a18/istio-proxy/0.log" Jul 24 16:26:16.533970 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:16.533939 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ddnwk_926abaa7-73a8-4e1e-827e-184ef782408b/discovery/0.log" Jul 24 16:26:16.905651 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:16.905615 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mdfxk_85e139bc-a01f-40c8-869d-c37b09ca7be4/istio-proxy/1.log" Jul 24 16:26:17.519160 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:17.519127 2560 scope.go:117] "RemoveContainer" containerID="77c396c401e71c221d1771fc0abb0ee74601e2378770d0983fe16bd54f10d4c2" Jul 24 16:26:24.238843 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:24.238800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:26:24.338978 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:24.338943 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" limit=1024 Jul 24 16:26:27.219154 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:27.219119 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hk86b_96502b52-d0fc-49f9-b839-8008f7b76266/global-pull-secret-syncer/0.log" Jul 24 16:26:27.364953 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:27.364905 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ldrdb_ff3a54e8-eddc-46d6-bec0-994078145a1b/konnectivity-agent/0.log" Jul 24 16:26:27.472656 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:27.472557 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-149.ec2.internal_fec91574dc3c4e18175b238dc0aeffc5/haproxy/0.log" Jul 24 16:26:33.234060 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:33.234020 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8psn9_98c8b36a-1f80-4e3a-925f-a372b30ff830/limitador/0.log" Jul 24 16:26:34.895344 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:34.895291 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4182968-f3dd-4f22-b9a1-a35b7e3ab797/alertmanager/0.log" Jul 24 16:26:34.919333 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:34.919288 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4182968-f3dd-4f22-b9a1-a35b7e3ab797/config-reloader/0.log" Jul 24 16:26:34.946739 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:34.946714 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4182968-f3dd-4f22-b9a1-a35b7e3ab797/kube-rbac-proxy-web/0.log" Jul 24 16:26:34.979423 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:34.979394 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4182968-f3dd-4f22-b9a1-a35b7e3ab797/kube-rbac-proxy/0.log" Jul 24 16:26:35.003282 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.003252 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4182968-f3dd-4f22-b9a1-a35b7e3ab797/kube-rbac-proxy-metric/0.log" Jul 24 16:26:35.026449 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.026403 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4182968-f3dd-4f22-b9a1-a35b7e3ab797/prom-label-proxy/0.log" Jul 24 16:26:35.053636 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.053607 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4182968-f3dd-4f22-b9a1-a35b7e3ab797/init-config-reloader/0.log" Jul 24 16:26:35.138884 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.138798 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-kfbt7_f47f7cdb-06a5-4319-a54d-4c6cb078e021/kube-state-metrics/0.log" Jul 24 16:26:35.158636 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.158609 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-kfbt7_f47f7cdb-06a5-4319-a54d-4c6cb078e021/kube-rbac-proxy-main/0.log" Jul 24 16:26:35.182524 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.182490 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-kfbt7_f47f7cdb-06a5-4319-a54d-4c6cb078e021/kube-rbac-proxy-self/0.log" Jul 24 16:26:35.236152 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.236125 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79bf54789c-s2xzw_d2dd966b-ad1e-4638-88b5-c002a015d6b4/metrics-server/0.log" Jul 24 16:26:35.280759 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.280721 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-bm9vz_44a77978-b18b-439a-88f1-f26c809e0eaf/monitoring-plugin/0.log" Jul 24 16:26:35.327848 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.327819 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8642_160eac8d-2761-4d6e-82bc-1af9ad1cd529/node-exporter/0.log" Jul 24 16:26:35.365421 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.365394 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8642_160eac8d-2761-4d6e-82bc-1af9ad1cd529/kube-rbac-proxy/0.log" Jul 24 16:26:35.429491 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.429409 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h8642_160eac8d-2761-4d6e-82bc-1af9ad1cd529/init-textfile/0.log" Jul 24 16:26:35.657551 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.657518 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s6mpr_d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a/kube-rbac-proxy-main/0.log" Jul 24 16:26:35.679847 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.679773 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s6mpr_d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a/kube-rbac-proxy-self/0.log" Jul 24 16:26:35.704173 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.704127 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-s6mpr_d08a2e9c-a4de-46ce-b7b7-56f20a1ce80a/openshift-state-metrics/0.log" Jul 24 16:26:35.757112 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.757082 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ed843ba5-2d7a-4904-afc5-860bdcb57859/prometheus/0.log" Jul 24 16:26:35.780833 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.780810 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ed843ba5-2d7a-4904-afc5-860bdcb57859/config-reloader/0.log" Jul 24 16:26:35.804844 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.804807 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ed843ba5-2d7a-4904-afc5-860bdcb57859/thanos-sidecar/0.log" Jul 24 16:26:35.826930 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.826905 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ed843ba5-2d7a-4904-afc5-860bdcb57859/kube-rbac-proxy-web/0.log" Jul 24 16:26:35.848184 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.848157 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ed843ba5-2d7a-4904-afc5-860bdcb57859/kube-rbac-proxy/0.log" Jul 24 16:26:35.869371 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.869343 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ed843ba5-2d7a-4904-afc5-860bdcb57859/kube-rbac-proxy-thanos/0.log" Jul 24 16:26:35.900527 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.900499 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ed843ba5-2d7a-4904-afc5-860bdcb57859/init-config-reloader/0.log" Jul 24 16:26:35.949328 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.949219 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-ttcjm_018ea29d-b885-434f-8108-eb5816b0e877/prometheus-operator/0.log" Jul 24 16:26:35.965833 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.965804 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-ttcjm_018ea29d-b885-434f-8108-eb5816b0e877/kube-rbac-proxy/0.log" Jul 24 16:26:35.993525 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:35.993498 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-5s89x_4423ad9d-f889-4187-bc2d-85566784ad61/prometheus-operator-admission-webhook/0.log" Jul 24 16:26:36.023682 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.023639 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6787bc55d6-9scqd_617fa295-bdfe-4dd1-a0ef-d161445e7092/telemeter-client/0.log" Jul 24 16:26:36.045571 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.045540 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6787bc55d6-9scqd_617fa295-bdfe-4dd1-a0ef-d161445e7092/reload/0.log" Jul 24 16:26:36.069103 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.069078 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6787bc55d6-9scqd_617fa295-bdfe-4dd1-a0ef-d161445e7092/kube-rbac-proxy/0.log" Jul 24 16:26:36.100517 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.100458 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fd8ff8b45-fr9xx_ca32c914-381b-4785-9954-30036cbd5537/thanos-query/0.log" Jul 24 16:26:36.122545 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.122508 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fd8ff8b45-fr9xx_ca32c914-381b-4785-9954-30036cbd5537/kube-rbac-proxy-web/0.log" Jul 24 16:26:36.148613 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.148590 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fd8ff8b45-fr9xx_ca32c914-381b-4785-9954-30036cbd5537/kube-rbac-proxy/0.log" Jul 24 16:26:36.169542 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.169515 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fd8ff8b45-fr9xx_ca32c914-381b-4785-9954-30036cbd5537/prom-label-proxy/0.log" Jul 24 16:26:36.190257 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.190232 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fd8ff8b45-fr9xx_ca32c914-381b-4785-9954-30036cbd5537/kube-rbac-proxy-rules/0.log" Jul 24 16:26:36.212617 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:36.212547 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fd8ff8b45-fr9xx_ca32c914-381b-4785-9954-30036cbd5537/kube-rbac-proxy-metrics/0.log" Jul 24 16:26:38.218359 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218163 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6"] Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218724 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218744 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218756 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218762 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218769 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218775 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218784 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218793 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218815 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218821 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218827 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" Jul 24 16:26:38.218827 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218832 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218838 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218844 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218852 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218857 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218863 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5684f10-86e2-4734-b566-185ad6634dad" containerName="revoke-keys" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218868 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5684f10-86e2-4734-b566-185ad6634dad" containerName="revoke-keys" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218878 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218883 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218890 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218898 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218955 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218962 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218968 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="882a76ba-6c28-4231-8080-6dab2bee7399" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218973 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa191de6-2595-4db3-abe1-3c73170d31fd" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218978 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="35838ff2-01d2-431f-9c63-20b6ac8791bd" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218986 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218991 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d35dd62d-9363-4637-ac0c-5afa422dbe31" containerName="istio-proxy" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.218995 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5684f10-86e2-4734-b566-185ad6634dad" containerName="revoke-keys" Jul 24 16:26:38.219234 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.219003 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="df6a470b-c088-4ae5-9597-7eba021b8cb5" containerName="istio-proxy" Jul 24 16:26:38.222241 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.222225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.224583 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.224554 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zsrwm\"/\"kube-root-ca.crt\"" Jul 24 16:26:38.224870 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.224851 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zsrwm\"/\"openshift-service-ca.crt\"" Jul 24 16:26:38.225607 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.225589 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zsrwm\"/\"default-dockercfg-cl78s\"" Jul 24 16:26:38.229912 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.229882 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6"] Jul 24 16:26:38.358124 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.358090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-lib-modules\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.358124 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.358126 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-sys\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.358376 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.358167 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-podres\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.358376 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.358213 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm6t2\" (UniqueName: \"kubernetes.io/projected/8b4d4d10-db89-427a-abea-a93d4306c96d-kube-api-access-xm6t2\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.358376 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.358245 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-proc\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.458984 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.458948 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-podres\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459201 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6t2\" (UniqueName: \"kubernetes.io/projected/8b4d4d10-db89-427a-abea-a93d4306c96d-kube-api-access-xm6t2\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459201 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-proc\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459201 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-podres\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459201 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459149 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-lib-modules\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459201 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-sys\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459201 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459187 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-proc\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459489 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459253 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-lib-modules\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.459489 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.459276 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b4d4d10-db89-427a-abea-a93d4306c96d-sys\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.467229 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.467205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm6t2\" (UniqueName: \"kubernetes.io/projected/8b4d4d10-db89-427a-abea-a93d4306c96d-kube-api-access-xm6t2\") pod \"perf-node-gather-daemonset-qfpx6\" (UID: \"8b4d4d10-db89-427a-abea-a93d4306c96d\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.533349 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.533296 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:38.669624 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:38.669590 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6"] Jul 24 16:26:38.670900 ip-10-0-132-149 kubenswrapper[2560]: W0724 16:26:38.670869 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8b4d4d10_db89_427a_abea_a93d4306c96d.slice/crio-cf802296b341d0f21b64e8b130d60d023db129319eed2a1ae1877197e70a32cd WatchSource:0}: Error finding container cf802296b341d0f21b64e8b130d60d023db129319eed2a1ae1877197e70a32cd: Status 404 returned error can't find the container with id cf802296b341d0f21b64e8b130d60d023db129319eed2a1ae1877197e70a32cd Jul 24 16:26:39.604024 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:39.603989 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" event={"ID":"8b4d4d10-db89-427a-abea-a93d4306c96d","Type":"ContainerStarted","Data":"be7202ab80ea852298fdad3eb3df0f1044ddf0426e891f09902441e60188db14"} Jul 24 16:26:39.604448 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:39.604039 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" event={"ID":"8b4d4d10-db89-427a-abea-a93d4306c96d","Type":"ContainerStarted","Data":"cf802296b341d0f21b64e8b130d60d023db129319eed2a1ae1877197e70a32cd"} Jul 24 16:26:39.604448 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:39.604076 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:39.621031 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:39.620984 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" podStartSLOduration=1.620968765 podStartE2EDuration="1.620968765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:39.619632549 +0000 UTC m=+3023.042233588" watchObservedRunningTime="2026-07-24 16:26:39.620968765 +0000 UTC m=+3023.043569794" Jul 24 16:26:40.569057 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:40.569021 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mqhw4_c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae/dns/0.log" Jul 24 16:26:40.588970 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:40.588936 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mqhw4_c98cd2a6-0d1a-4be1-9a64-e5331e27c0ae/kube-rbac-proxy/0.log" Jul 24 16:26:40.669258 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:40.669229 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbwxh_b556c24d-9aee-4fd8-ab2c-9605f1300488/dns-node-resolver/0.log" Jul 24 16:26:41.296913 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:41.296881 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-76tzw_105c143a-fbc6-4945-8311-5e9be7b5ce68/node-ca/0.log" Jul 24 16:26:42.363851 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:42.363818 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp_df4dfeac-3176-418d-a129-3438754d8a18/istio-proxy/0.log" Jul 24 16:26:42.635967 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:42.635876 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ddnwk_926abaa7-73a8-4e1e-827e-184ef782408b/discovery/0.log" Jul 24 16:26:42.714604 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:42.714548 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mdfxk_85e139bc-a01f-40c8-869d-c37b09ca7be4/istio-proxy/0.log" Jul 24 16:26:42.767985 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:42.767945 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mdfxk_85e139bc-a01f-40c8-869d-c37b09ca7be4/istio-proxy/1.log" Jul 24 16:26:43.708126 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:43.708094 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dj77_1e689b39-eaaf-4e95-86f9-9018172c2f7c/serve-healthcheck-canary/0.log" Jul 24 16:26:44.327054 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:44.327016 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84w5l_c4eafb17-8bbc-42ec-8284-0330af851b55/kube-rbac-proxy/0.log" Jul 24 16:26:44.353263 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:44.353227 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84w5l_c4eafb17-8bbc-42ec-8284-0330af851b55/exporter/0.log" Jul 24 16:26:44.376418 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:44.376395 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84w5l_c4eafb17-8bbc-42ec-8284-0330af851b55/extractor/0.log" Jul 24 16:26:45.619451 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:45.619423 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-qfpx6" Jul 24 16:26:48.006216 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:48.006182 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-n54rh_39ac11c7-c954-4499-8b76-fac6954bc85d/manager/0.log" Jul 24 16:26:48.024723 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:48.024690 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-n54rh_39ac11c7-c954-4499-8b76-fac6954bc85d/copy-manifests/0.log" Jul 24 16:26:49.367590 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:49.367530 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp_df4dfeac-3176-418d-a129-3438754d8a18/istio-proxy/0.log" Jul 24 16:26:49.473132 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:49.473069 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7db5d47bff-pngv2_5b021708-a630-4b5a-bfa9-c25fcc81a93d/manager/0.log" Jul 24 16:26:49.636428 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:49.636332 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ddnwk_926abaa7-73a8-4e1e-827e-184ef782408b/discovery/0.log" Jul 24 16:26:49.698232 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:49.698200 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mdfxk_85e139bc-a01f-40c8-869d-c37b09ca7be4/istio-proxy/0.log" Jul 24 16:26:49.754117 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:49.754085 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mdfxk_85e139bc-a01f-40c8-869d-c37b09ca7be4/istio-proxy/1.log" Jul 24 16:26:51.210015 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:51.209986 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-chnfp_df4dfeac-3176-418d-a129-3438754d8a18/istio-proxy/0.log" Jul 24 16:26:51.489809 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:51.489726 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ddnwk_926abaa7-73a8-4e1e-827e-184ef782408b/discovery/0.log" Jul 24 16:26:51.538228 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:51.538189 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mdfxk_85e139bc-a01f-40c8-869d-c37b09ca7be4/istio-proxy/0.log" Jul 24 16:26:51.593632 ip-10-0-132-149 kubenswrapper[2560]: I0724 16:26:51.593597 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mdfxk_85e139bc-a01f-40c8-869d-c37b09ca7be4/istio-proxy/1.log"